Developing a checklist for reporting research using simulated patient methodology (CRiSP): a consensus study

Evaluation of Medical Subject Headings assignment in simulated patient articles

OBJECTIVES

To evaluate human-based Medical Subject Headings (MeSH) allocation in articles about 'patient simulation'-a technique that mimics real-life patient scenarios with controlled patient responses.

METHODS

A validation set of articles indexed before the Medical Text Indexer-Auto implementation (in 2019) was created with 150 combinations potentially referring to 'patient simulation'. Articles were classified into four categories of simulation studies. Allocation of seven MeSH terms (Simulation Training, Patient Simulation, High Fidelity Simulation Training, Computer Simulation, Patient-Specific Modelling, Virtual Reality, and Virtual Reality Exposure Therapy) was investigated. Accuracy metrics (sensitivity, precision, or positive predictive value) were calculated for each category of studies.

KEY FINDINGS

A set of 7213 articles was obtained from 53 different word combinations, with 2634 excluded as irrelevant. 'Simulated patient' and 'standardized/standardized patient' were the most used terms. The 4579 included articles, published in 1044 different journals, were classified into: 'Machine/Automation' (8.6%), 'Education' (75.9%) and 'Practice audit' (11.4%); 4.1% were 'Unclear'. Articles were indexed with a median of 10 MeSH (IQR 8-13); however, 45.5% were not indexed with any of the seven MeSH terms. Patient Simulation was the most prevalent MeSH (24.0%). Automation articles were more associated with Computer Simulation MeSH (sensitivity = 54.5%; precision = 25.1%), while Education articles were associated with Patient Simulation MeSH (sensitivity = 40.2%; precision = 80.9%). Practice audit articles were also polarized to Patient Simulation MeSH (sensitivity = 34.6%; precision = 10.5%).

CONCLUSIONS

Inconsistent use of free-text words related to patient simulation was observed, as well as inaccuracies in human-based MeSH assignments. These limitations can compromise relevant literature retrieval to support evidence synthesis exercises.

Counselling with a focus on product and price transparency for over-the-counter headache medicines: A simulated patient study in community pharmacies in Munich, Germany

BACKGROUND

In Germany, over-the-counter (OTC) medicines may only be dispensed by community pharmacies (CPs). German CPs must ensure 'adequate' counselling, including the cost of medicines. Along with information gathering and advice giving as classic aspects of counselling, the aim was also to investigate counselling indicators of product and price transparency.

METHODS

The cross-sectional study was based on the covert simulated patient (SP) methodology and was conducted in a random sample of CPs stratified by districts in the major German city of Munich. Each of the 178 selected CPs was visited once by one of five trained female students. They simulated a symptom-based sub-scenario 1 with a request for an OTC medicine for a headache and a sub-scenario 2 with standardised information regarding product and price transparency. The assessment, completed immediately postvisit by the SPs, included a total of 23 items.

RESULTS

All 178 scheduled visits were completed successfully. The median counselling score with the classic items was 3.0 out of 12 points (interquartile range [IQR] 4.25) and when expanded by items for product and price transparency the score was 4.0 out of 14 points (IQR 4.00). A selection of medicines was offered unsolicited in 38.2% of the visits and in 5.6% of the visits voluntary price information was provided before the transaction. A request for a cheaper medicine led to a significant price reduction (Wilcoxon signed-rank test; p < 0.001, r = 0.869).

CONCLUSION

Due to the below-average level of counselling, the regional chambers of pharmacists are recommended to initiate measures for improvement. There is also potential for optimisation with regard to product and price transparency as an important extension of the classic counselling aspects. It is therefore recommended that the government raise customers' awareness of the cost of medicines.

Exploring the usability of simulated patient methodology in dental clinics in Western Australia: A pilot survey

OBJECTIVES

This study aimed to explore the dental staff knowledge of simulated patient methodology and support for its use to investigate dental staffs' triaging ability.

MATERIAL AND METHODS

Staff at dental practices in Western Australia were invited to participate in a cross-sectional online questionnaire, consisting of demographic questions, questions on triaging, and knowledge of simulated patient methodology. Descriptive and parametric tests were undertaken for quantitative data; qualitative responses were thematically analyzed.

RESULTS

Of the 100 participants, most were female (71%), aged 25-39 years (57%), dentists (46%), and worked in private practices (60%). While 82% of participants triaged dental appointment enquiries, only 26% had heard of simulated patient studies. The majority (66%) of participants spent 1-5 min when triaging appointments and less than half (29%) asked about medical history, aggravating or alleviating factors. Although there was a general positive attitude toward use of simulated patient methodology to investigate practice, some concerns were identified.

CONCLUSIONS

The findings of our exploratory study suggests that there may be a potential for utilizing simulated patient studies to improve the care of patients by dental receptionists in general dental practices.

Refining the CRiSPHe (checklist for reporting research using a simulated patient methodology in Health): a Delphi study

OBJECTIVES

A recent applicability study highlighted the need for the existing checklist for reporting research using a simulated patient methodology (CRiSP) to be clearer and user-friendly. The aim of this study was to update the checklist to address these concerns.

METHODS

A fourth round of the Delphi consensus study, used in the original checklist development work, was conducted. Previous participants, who had expertise in SP methodology, were invited to complete a questionnaire including a list of 13 checklist items developed in the previous study and revised following applicability testing. Closed questions were analysed for frequency. Consensus was predefined as >80% agreement. All items were discussed in a roundtable meeting and further modified as necessary. Responses to open questions were content analysed.

KEY FINDINGS

Twenty-one authors participated. There was a statistical consensus in 12 out of 13 modified checklist items.

CONCLUSIONS

A final reporting checklist for studies in health research using SP methodology has been developed using a consensus approach. Further refinements may be needed to increase the generalizability of the checklist in different contexts.

Simulated patient methodology as a "gold standard" in community pharmacy practice: Response to criticism

The simulated patient methodology (SPM) is considered the "gold standard" as covert participatory observation. SPM is attracting increasing interest for the investigation of community pharmacy practice; however, there is criticism that SPM can only show a small picture of everyday pharmacy practice and therefore has limited external validity. On the one hand, a certain design and application of the SPM goes hand in hand with an increase in external validity. Even if, on the other hand, this occurs at the expense of internal validity due to the trade-off situation, the justified criticism of the SPM for investigating community pharmacy practice can be countered.

Quality of Refractive Error Care in Cambodia: An Unannounced Standardized Patient Study

PURPOSE

Quality-of-care in refractive error services is essential, as it directly affects vision outcomes, wellbeing, educational attainment, and workforce participation. In Cambodia, uncorrected refractive error is a leading cause of mild and moderate vision impairment in adults. We evaluated the quality of refractive error care in Cambodia by estimating the proportion of prescribed and dispensed spectacles appropriate for people's refractive error needs and factors associated with spectacle quality.

METHODS

A cross-sectional protocol was employed with 18 Khmer-speaking adult participants observing testing procedures in 156 optical services across six provinces in 2022. A total of 496 dispensed spectacles were assessed against spectacle quality indicators.

RESULTS

The analysis revealed that 35.1% of dispensed spectacles were of optimal quality. The most common error observed in sub-optimal spectacles was the presence of horizontal prism outside of tolerance limits. The study also found that 44.0% of emmetrope visits involved unnecessary prescription spectacle recommendations, and 18.3% of written prescriptions did not correspond with dispensed spectacles. Sex differences were observed, with men predominantly providing refractive error care and women more likely to be unnecessarily recommended prescription spectacles.

CONCLUSION

The findings highlight the importance of prioritizing quality-of-care in refractive error services. A key recommendation is to consider regulatory mechanisms to ensure optical services employ appropriately qualified staff. Additionally, efforts should be made to eliminate unnecessary prescriptions -- especially for emmetropes and females -- standardize written prescriptions, ensure consistent pupil distance measurements, reduce reliance on autorefraction, and address the gender imbalance in the refractive error workforce.

Quality of refractive error care in Pakistan: an unannounced standardised patient study

OBJECTIVE

Undercorrected refractive errors are the primary cause of vision impairment worldwide, including in Pakistan. However, limited data exist on the quality of refractive error care. Our study assessed the quality of refractive error care in Punjab, Pakistan by estimating the proportion of spectacles that were optimally prescribed.

METHODS AND ANALYSIS

In this cross-sectional study, 12 unannounced standardised patients (USPs) from Jhang, Khanewal and Sahiwal districts were recruited. USPs underwent baseline subjective refraction and were trained to attend optical services, observe consultations, request spectacles and obtain prescriptions. The spectacles received were compared with baseline refraction to determine quality. We also examined the associations between spectacle quality, service and patient characteristics.

RESULTS

Out of 276 attempted visits to 69 optical services, 241 pairs of spectacles were dispensed. A population size-weighted percentage of 42.7% (95% CI 36.4% to 49.3%) of spectacles were optimal quality, with the range varying from 13.8% in Jhang to 67.0% in Khanewal. Half the suboptimal quality spectacles had horizontal prism deviations outside of tolerance limits. Optimal spectacles were associated with performing focimetry (unadjusted OR=7.15, 95% CI (3.02 to 16.94), p<0.001) and good communication (OR=2.23, 95% CI (1.06 to 4.67), p=0.03). Hyperopic USPs were less likely to receive optimal spectacles (OR=0.01 95% CI (0.00 to 0.11), p<0.001).

CONCLUSION

The quality of refractive error care in Pakistan requires improvement, particularly in the Jhang district. Key areas for enhancing refractive error care in Pakistan include refining dispensing and refraction skills for hyperopic prescriptions, providing training on the risks of using previous spectacles, and emphasising the importance of effective communication skills.

Oral emergency contraception practices of community pharmacies: a mystery caller study in the capital of Germany, Berlin

BACKGROUND

In Germany, oral emergency contraception (EC) with the active ingredients levonorgestrel (LNG) and ulipristal acetate (UPA) is available as over-the-counter (OTC) medicine only from community pharmacies (CPs). Because of the window of effect, which is limited to only a few days, CPs have a great responsibility to provide rapid and unimpeded access, while also ensuring "adequate" counseling. The aim was-for the first time in Europe and thus also in Germany for the methodology used in this study-to investigate immediate availability, pricing, and aspects of counseling.

METHODS

Covert mystery calls were conducted in a random sample of CPs stratified by districts in the German capital Berlin. Each of the 263 CPs included was called once at random by one of two trained female student mystery callers. They simulated a product-based scenario for the UPA original ellaOne^®, citing a contraceptive failure one day ago as the reason.

RESULTS

Of 257 successfully called CPs, UPA preparations were immediately available in 98.4% (253/257) and LNG preparations in 86.8% (184/212) of CPs. Prices for UPA preparations varied from €15.95 to €42.95 (∆ 169%; median €35.00 [interquartile range (IQR) €5.91]) and for LNG preparations from €10.60 to €32.49 (Δ 207%; median €22.00 [IQR €5.76]). Information about the correct different window of effect of UPA and LNG preparations was provided in 69.8% (127/182) of CPs. UPA preparations were recommended in 63.1% (111/176) and LNG preparations in 17.2% (30/174) of CPs. Information was provided on how to take them as soon as possible in 30.8% (44/143) of CPs and on how to use them after vomiting in 46.0% (64/139).

CONCLUSIONS

Berlin CPs support access through high immediate availability, especially to UPA preparations. However, access is hampered by high absolute price ranges of both UPA and LNG preparations, which could ideally be minimized by a comparison app. It is positive that CPs promote the benefits of UPA preparations by recommending them noticeably more often than LNG preparations. However, there are deficiencies in giving advice, so there is a need to raise awareness among pharmacy staff to ensure "adequate" counseling in advance over the phone.

Using the Simulated Patient Methodology in the Form of Mystery Calls in Community Pharmacy Practice Research: A Scoping Review Protocol

Community pharmacies (CPs) play a major role in health care delivery. The simulated patient methodology (SPM), which is considered the "gold standard", is recommended for studying CP practice. SPM can be applied in different forms, which include visits and also calls ("mystery calls"). So far, only the role of visits in the investigation of CP practice is known. As the first study worldwide, a systematic map of such reports will be provided, which applies calls in the context of the SPM for the study of CP practice. Reports with the pharmacy staff as the population under study should be included. Reports should be included that conduct an investigation using the SPM in the form of calls to simulate patients and other customers. Reports should be included that examine CP practice as defined by the International Pharmaceutical Federation and the World Health Organization (FIP/WHO). The scoping review methodology will be applied using the associated guidelines from Joanna Briggs Institute (JBI) and PRISMA extension for Scoping Review (PRISMA-ScR). The search will be for both published and unpublished original research in English with transparent information on SPM until the end of 2022. The plan is to search Embase, MEDLINE via PubMed, and Web of Science. Directly afterward, the respective literature collection of the reviewers and the reference lists of suitable international reviews will be searched. This will be followed by a forward and backward snowballing in Google Scholar. For the inclusion of reports, a selection process and for the data extraction a data charting process with the help of variables derived from related reviews and from two SPM-form spanning international guidelines will be performed. The data extracted from the included reports should be synthesized and presented in MS Excel tabular form using the previously determined variables.

Evaluation of pharmacist's practices regarding the antimicrobials dispensing: a simulated patient study

BACKGROUND

The indiscriminate use of antimicrobials is considered a major contributing factor to the increase in antimicrobial resistance. Community pharmacies are the main source of access to antimicrobials, and pharmacists are in a strategic position to promote rational use of these medicines. Therefore, it is important to know dispensing service quality.

OBJECTIVE

To evaluate the behavior of pharmacists in dispensing antimicrobials in community pharmacies in northeast Brazil.

METHODS

This cross-sectional pilot study was conducted from August to October 2021 in a private community pharmacy chain in Sergipe. Dispensing was evaluated using the simulated patient (SP) technique. Two SP asked the pharmacists for the antimicrobials (case clinic 1: upper respiratory infection; case clinic 2: urinary tract infection) and recorded the service through audio. Dispensing practices were independently analyzed by two researchers based on the tools available in the literature. Data were presented using descriptive statistics.

RESULTS

A total of 54 simulated patient visits were conducted. Based on the 12 steps recommended by the research team for good dispensing, pharmacists asked an average of 1 (±1.17) question for upper respiratory infections and 0.3 (±0.54) for urinary tract infections, as well as provided counseling (mean number of recommendations, 2.6 (±1.44) and 4.5 (±2.35), respectively). As for communication skills, pharmacists had a regular score (3.07 ± 0.34). Furthermore, there was no significant difference in the number of steps and counseling recommendations by pharmacists in dispensing clinical cases 1 and 2 (p = 0.0674).

CONCLUSION

The quality of antimicrobial dispensing was evaluated as suboptimal, requiring improvements in practice and multifaceted strategies to promote continuing education of these professionals. In addition, awareness actions for the population must be implemented to promote the rational use of antimicrobials and reduce microbial resistance.

Patient and Family Representation in Randomized Clinical Trials Published in 3 Medical and Surgical Journals: A Systematic Review

IMPORTANCE

Patient and family engagement in research may improve the design, conduct, and dissemination of clinical research, but little is known about whether these stakeholder groups are involved in the design and conduct of randomized clinical trials.

OBJECTIVE

To characterize the involvement and role of patient and family representatives in the design and conduct of randomized clinical trials by reviewing randomized clinical trials from 3 peer-reviewed medical and surgical journals with high impact factors.

EVIDENCE REVIEW

In this systematic review, the first 50 consecutive randomized clinical trials published on or after January 1, 2021, until September 30, 2021, from each of 3 medical and surgical journals with high impact factors were reviewed for patient or family involvement in trial design and/or conduct. The manuscript, supplemental data, and trial registry records were searched for trial design and governance structures. Two independent, blinded reviewers screened citations and extracted data. This study followed the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) guidelines.

FINDINGS

Only 7 of 150 randomized clinical trials (5%) reported patient or family representation in their study design or conduct. Most studies with patient or family representation (n = 5) were from a single journal. Stakeholder involvement was mainly in the execution phase (n = 7), although in 2 studies stakeholders were also involved in the translation phase.

CONCLUSIONS AND RELEVANCE

The findings of this systematic review suggest that patient or family involvement in the design and conduct of randomized clinical trials in the publications with high impact factors is lacking. We found that when patient or family groups are involved in research, the focus was mainly on the execution phase of research design. There is a need to increase stakeholder involvement in the research design, conduct, and translation of randomized clinical trials.

The Quality of Counseling for Headache OTC Medications in German Community Pharmacies Using a Simulated Patient Approach: Are There Differences between Self-Purchase and Purchase for a Third Party?

Background

In Germany-as worldwide-headache is one of the most frequent causes of self-medication. The dispensing of over-the-counter (OTC) medications may only be carried out by community pharmacies (CPs). In doing so, CPs have to ensure "adequate" counseling, for both self-purchase and purchase for a third party, which also occurs in everyday pharmacy practice. The aim of this study was to evaluate the quality of counseling for headache OTC medications in German CPs and, as the first study worldwide, to analyze whether and to what extent there are differences in counseling between self-purchase and purchase for a third party.

Methods

A cross-sectional study was carried out using the covert simulated patient methodology (SPM) in all 42 CPs in the German big city Potsdam. With the help of 8 trained simulated patients (SPs), each CP was visited four times by a different SP. The SPs simulated in each CP two scenarios two times with the demand for an OTC medication against headache, which differed only in whether the demand was for themselves or for their boyfriend/girlfriend.

Results

All 168 planned pharmacy visits (84 visits per scenario) were successfully carried out. Overall, the median counseling score was 3.0 out of 9 points (interquartile range [IQR 2.0]). There were no significant differences between the two scenarios (Wilcoxon signed-rank test; p=0.495, r = 0.053). In a multivariate binary logistic regression analysis, the counseling level and the different scenarios were not significantly associated (adjusted odds ratio [AOR] = 1.635, 95% CI = 0.673-3.972, p=0.278).

Conclusions

Due to the partly considerable deficits in counseling of German CPs, policy-makers and the regional chambers of pharmacists are called upon to take appropriate measures to improve the quality of counseling. It is positive that no differences in counseling between self-purchase and purchase for a third party were found, but further verifying studies with a modified methodology are recommended.

Assessing content and factors influencing responses to information requests in community pharmacies in Jordan: A simulated patients study

OBJECTIVES

To assess the management of requests for information about a prescription only medicine (simvastatin for treatment of dyslipidemia) by pharmacy staff in community settings and explore the factors influencing the information content.

METHODS

A cross sectional study conducted using the stimulated patient (SP) method between November 2018 and May 2019. The SP conveyed the request at the beginning of the encounter in a standardized way based on predetermined plots and was instructed to ask the pharmacy staff directly if information was not discussed spontaneously. After the visit, the SP provided written feedback including information about the scenario and a copy of individualized feedback. The study was reported according to the checklist for reporting research using simulated patient methodology (CRiSP). Factors influencing information content with or without information demand were investigated.

RESULTS

A total of 55 visits were analyzed. The average content score for the information discussed spontaneously was 16.2% with the standard deviation (SD) equal to15.6. The score improved significantly after information was demanded by the SP; the average total information content score became 34.4% (SD = 16) with p < 0.001. The score of information discussed spontaneously was higher for male pharmacy staff, older age, more experience, and a Pharm D degree. When the SP prompted or demanded for information, older pharmacy staff with more experience and with a college degree scored higher. Independent pharmacies, longer visit durations, and less distraction were associated significantly with higher information scores Pharmacy staff aged 35-39 and those with 6-10 years of experience were significant contributors to spontaneous discussion of information with p values = 0.003 and 0.013, respectively. After the SP demanded information, pharmacy staff with less than 5 years of experience and greater confidence as well as longer visits were positively predicting higher information scores with p values of 0.049, 0.04, and 0.04, respectively.

CONCLUSIONS

Information provided by community pharmacists responding to information requests about prescription only medications was found to be suboptimal. Patient requests for information were found to be a positive driver for better information content. Further research of mixed methodologies is required to clarify the factors and motivators enabling information exchange in community settings and to outline true training needs.

Exploring hospital compliance with the primary nursing care model: validating an inventory using the Delphi method

Background

The primary nursing care model is considered a personalized model of care delivery based on care continuity and on the relationship between the nurse and patient. Primary nursing checklists are not often mentioned in the literature; however, they represent a valid instrument to develop, implement, and evaluate primary nursing. The aim of this study was to create a structured checklist to explore hospital compliance in primary nursing.

Methods

The Delphi method was used to develop and validate a checklist. The preliminary version was created and sent to three experts for their opinions. Their comments were ultimately used in the first version, which included four components with 19 items regarding primary nursing characteristics. A two-round Delphi process was used to generate consensus items. The Delphi panel consisted of six experts working in primary nursing contexts and/or teaching or studying primary nursing. Data were collected using a structured questionnaire from July 2020 to January 2021. These experts were asked to rate each element for relevance using a 4-point Likert-type scale. Furthermore, the consensus among the panel of experts was set at ≥78%, with selected items being voted “quite relevant” and “highly relevant”. Content validity index (I-CVI) and modified kappa statistic were also calculated. Following expert evaluation, the first version of the checklist was modified, and the new version, constituting 17 items, was sent to the same experts.

Results

The first version of the checklist demonstrated a main relevance score of 3.34 (SD = 0.83; range = 1.3–4; mean I-CVI = 0.84; range: 0.83–1), but three items did not receive an adequate I-CVI score, that is, lower than 0.78. After the second round, the I-CVIs improved. The main score of relevance was 3.61 (SD: 0.35; range = 2.83–4; mean = I-CVI: 0.93). The S-CVI/UA was 0.58, and the S-CVI/Ave was 0.93.

Conclusion

Measuring primary nursing compliance should be implemented to provide continuous feedback to nurses. Moreover, utilizing valid checklists could permit comparing different results from others’ research. Future research should be conducted to compare the results from the checklist with nursing outcomes.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12912-021-00712-1.

Dispensing and Variabilities in Pricing of Headache OTC Medicines by Community Pharmacies in a German Big City: A Simulated Patient Approach

Background

In Germany, over-the-counter (OTC) medicines for self-medication may only be dispensed by community pharmacies (CPs). From the customer's point of view, "adequate" counseling includes not only the recommendation of medicines that meet guidelines, but ao the dispensing of low-priced medicines. This is all the more important in Germany against the background of free pricing and a lack of obligation to display prices. The aim was to analyze the dispensing and possible price variability of OTC medicines for headache.

Methods

The cross-sectional design of the study based on the simulated patient methodology (SPM) included all 42 CPs of the German big city Potsdam and used eight trained simulated patients (SPs). Between October and December 2020, four different SPs visited each CP four times with two almost identical scenarios with the demand for a medicine for headache.

Results

All 168 planned visits were successfully completed and ibuprofen (60.1%) was dispensed most frequently. Across all dispensed medicines, prices varied from €0.93 to €9.97 (∆ 972%; median €3.46 [interquartile range (IQR) €2.25]). For repeated dispensing of the same CPs, the price variability was a maximum of €8.77. In addition, packs with ≤10 tablets were dispensed in 47.6%, with 11 to 20 tablets in 50.6%, and with >20 tablets in 1.8% of visits.

Conclusion

Increasing price transparency and strengthening price competition could make it easier for customers to access OTC medicines according to their personal needs.