Understanding negative feedback from South Asian patients: an experimental vignette study

Creating and administering video vignettes for a study examining the communication of diagnostic uncertainty: methodological insights to improve accessibility for researchers and participants

BACKGROUND

Studying clinician-patient communication can be challenging, particularly when research seeks to explore cause-and-effect relationships. Video vignettes - hypothetical yet realistic scenarios - offer advantages to traditional observational approaches by enabling standardisation and manipulation of a clinician-patient encounter for assessment by participants. While published guidelines outline stages to create valid video vignette studies, constructing high quality vignettes which are accessible to a wide range of participants and feasible to produce within time and budget restraints remains challenging. Here, we outline our methods in creating valid video vignettes to study the communication of diagnostic uncertainty. We aim to provide practically useful recommendations for future researchers, and to prompt further reflection on accessibility issues in video vignette methodology.

METHODS

We produced four video vignettes for use in an online study examining the communication of diagnostic uncertainty. We followed established guidelines for vignette production, with specific consideration of how these might be applied pragmatically to save time and resources. Scripts were pilot-tested with 15 laypeople, and videos with 14 laypeople; pilot-testing involved both quantitative and qualitative analysis.

RESULTS AND DISCUSSION

We demonstrate the usefulness of existing guidelines, while also determining that vignette production need not necessarily be expensive or time-consuming to be valid. Our vignettes were filmed using an iPhone camera, and featured a physician rather than a professional actor; nonetheless, pilot-testing found them to be internally and externally valid for experimental use. We thus propose that if care is taken in initial script development and if pragmatic choices are made regarding filming techniques and pilot-testing, researchers can produce valid vignettes within reasonable time and budget restraints. We also suggest that existing research fails to critically examine the potential benefits and harms of online video vignette methodology, and propose that further research should consider how it can be adapted to be inclusive of those from underserved backgrounds.

CONCLUSIONS

Researchers creating video vignette studies can adapt the video vignette development process to suit time and budget constraints, and to make best use of available technology. Online methods may be harnessed to increase participant accessibility, but future research should explore more inclusive vignette design.

Comparing healthcare quality: A common framework for both ordinal and cardinal data with an application to primary care variation in England

The paper proposes a framework for comparing the quality of healthcare providers and assessing the variation in quality between them, which is directly applicable to both ordinal and cardinal quality data on a comparable basis. The resultant measures are sensitive to the full distribution of quality scores for each provider, not just the mean or the proportion meeting some binary quality threshold, thereby making full use of the multicategory response data increasingly available from patient experience surveys. The measures can also be standardized for factors such as age, sex, ethnicity, health and deprivation using a distribution regression model. We illustrate by measuring the quality of primary care services in England in 2019 using three different sources of publicly available, general practice-level information: multicategory response patient experience data, ordinal inspection ratings and cardinal clinical achievement scores. We find considerable variation at both local and regional levels using all three data sources. However, the correlation between the comparative quality indices calculated using the alternative data sources is weak, suggesting that they capture different aspects of general practice quality.

The use of experimental vignette studies to identify drivers of variations in the delivery of health care: a scoping review

Background

Identifying how unwarranted variations in healthcare delivery arise is challenging. Experimental vignette studies can help, by isolating and manipulating potential drivers of differences in care. There is a lack of methodological and practical guidance on how to design and conduct these studies robustly. The aim of this study was to locate, methodologically assess, and synthesise the contribution of experimental vignette studies to the identification of drivers of unwarranted variations in healthcare delivery.

Methods

We used a scoping review approach. We searched MEDLINE, Embase, Web of Science and CINAHL databases (2007–2019) using terms relating to vignettes and variations in healthcare. We screened title/abstracts and full text to identify studies using experimental vignettes to examine drivers of variations in healthcare delivery. Included papers were assessed against a methodological framework synthesised from vignette study design recommendations within and beyond healthcare.

Results

We located 21 eligible studies. Study participants were almost exclusively clinicians (18/21). Vignettes were delivered via text (n = 6), pictures (n = 6), video (n = 6) or interactively, using face-to-face, telephone or online simulated consultations (n = 3). Few studies evaluated the credibility of vignettes, and many had flaws in their wider study design. Ten were of good methodological quality. Studies contributed to understanding variations in care, most commonly by testing hypotheses that could not be examined directly using real patients.

Conclusions

Experimental vignette studies can be an important methodological tool for identifying how unwarranted variations in care can arise. Flaws in study design or conduct can limit their credibility or produce biased results. Their full potential has yet to be realised.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12874-021-01247-4.

Are parents' geographical origin associated with their evaluation of child and adolescent mental health services? Results from a national survey in Norway

The objective was to assess the association between parents' geographical origin and their evaluation of outpatient child and adolescent mental health services (CAMHS). Data were collected in a national parent's experience survey of all outpatient CAMHS in Norway in 2017. Following exclusions, 16,143 parents were part of the study, of which 5932 responded (36.1%). Diagnosis and global psychosocial functional level were collected from the National Patient Register. Multilevel regression was used to assess the association between parents' geographical origin and parent evaluation of the outpatient CAMHS on ten indicators. Sentiment and content analysis was conducted on open-ended comments from parents. The estimated regression coefficients showed that parents born in Eastern Europe scored the services significantly poorer than parents born in Norway on outcome of treatment (- 7.73, p < 0.01), general satisfaction (- 5.53, p < 0.05), ease of getting in contact with health personnel outside of scheduled appointments (- 17.04, p < 0.001), and knowledge of the services that the child has received at the service (- 10.63, p < 0.001). Parents born in Asia/Africa/South America scored the services similar as Norwegian parents on eight of ten indicators, better on one (waiting time) and poorer on one (ease of getting in contact). Sentiment analysis showed that 54% of the comments from parents born in Eastern Europe were negative, compared to 42% for the Norwegian group and 36% for Asia/Africa/South America. The parents' evaluation of the outpatient CAMHS were partly associated with their geographical origin, with parents born in Eastern Europe reporting poorer experiences than parents born in Norway.

Patient experiences with general practice in Norway: a comparison of immigrant groups and the majority population following a national survey

Background

Patient experience is an important indicator of quality of health care. In Norway, little is known about the quality of health care for immigrants. The aim of this study was to compare patient-reported experiences with general practice between the Norwegian-born population and immigrant groups.

Methods

We performed secondary analyses of data from a national survey on patient experiences with general practice, including assessments of general practitioners (GPs) and their GP offices. The survey was carried out in Norway in 2018–19. The total number of respondents was 2029, with a response rate of 42.6%. Region of birth was available for 1981 participants, and these were included in the analyses (“Norway” (N = 1756), “Asia, Africa or South America” (N = 95), “Eastern Europe” (N = 70) and “Western Europe, North America or Oceania” (N = 60)). Five indicators of patient experiences were used as dependent variables in bivariate and multivariate analyses, with region of birth as the main exposure variable and other background variables about the patient as adjustment variables: “the GP” (measures related to communication and competency), “auxiliary staff” (politeness, competency, organization), “accessibility” (waiting times), “coordination” (with other services) and `enablement` (GP facilitates coping with/understanding illness).

Results

Immigrants as a whole reported poorer experiences with general practice than the majority population, with significantly poorer scores on four of five patient experience indicators. Patients from Asia/Africa/South America reported poorer experiences than those from Norway on the indicators “GP”, “auxiliary staff”, “accessibility” and “coordination”: on a scale from 0 to 100 where 100 is the best, the difference ranged from 7.8 (GP) to 20.3 (accessibility). Patients from Eastern Europe reported lower scores on “GP” and patients from Western Europe/North America/Oceania reported lower scores on “auxiliary staff”. These associations were still significant after adjustment for sex, age, self-rated physical and mental health, number of contacts with the GP and education.

Conclusions

For countries with a substantial proportion of foreign-born patients in the health system, immigrant background is an important parameter in quality improvement work. Immigrant background is also an important parameter in health service research.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12913-020-05963-3.

Applicant Fairness Perceptions of a Robot-Mediated Job Interview: A Video Vignette-Based Experimental Survey

It is well-established in the literature that biases (e. g., related to body size, ethnicity, race etc.) can occur during the employment interview and that applicants' fairness perceptions related to selection procedures can influence attitudes, intentions, and behaviors toward the recruiting organization. This study explores how social robotics may affect this situation. Using an online, video vignette-based experimental survey (n = 235), the study examines applicant fairness perceptions of two types of job interviews: a face-to-face and a robot-mediated interview. To reduce the risk of socially desirable responses, desensitize the topic, and detect any inconsistencies in the respondents' reactions to vignette scenarios, the study employs a first-person and a third-person perspective. In the robot-mediated interview, two teleoperated robots are used as fair proxies for the applicant and the interviewer, thus providing symmetrical visual anonymity unlike prior research that relied on asymmetrical anonymity, in which only one party was anonymized. This design is intended to eliminate visual cues that typically cause implicit biases and discrimination of applicants, but also to prevent biasing the interviewer's assessment through impression management tactics typically used by applicants. We hypothesize that fairness perception (i.e., procedural fairness and interactional fairness) and behavioral intentions (i.e., intentions of job acceptance, reapplication intentions, and recommendation intentions) will be higher in a robot-mediated job interview than in a face-to-face job interview, and that this effect will be stronger for introvert applicants. The study shows, contrary to our expectations, that the face-to-face interview is perceived as fairer, and that the applicant's personality (introvert vs. extravert) does not affect this perception. We discuss this finding and its implications, and address avenues for future research.

The effects of essentialist thinking toward biosocial risk factors for criminality and types of offending on lay punishment support

This research uses experimental methods to gauge how different facets of essentialist thinking toward (1) types of offending and (2) biosocial risk factors for criminality predict lay punishment support. A randomized between-subjects experiment using contrastive vignettes was conducted with members of the general public (N = 897). Overall, as hypothesized, aspects of essentialist thinking, particularly informativeness, continuity, immutability, and discreteness, toward both biosocial risk factors and types of offending behavior generally predicted more severe punishments surrounding retribution, incapacitation, and deterrence. Yet, surprisingly, several of the same beliefs, specifically toward discreteness and informativeness, also predicted non-punitive sentiments toward restoration and decreased prison time in some contexts. This work demonstrates that essentialist thinking not only may affect how the public cognitively categorizes biosocial risk factors for criminality and types of offending, but also may have consequences for public support for the punishment of offenders with particular offense records or characteristics.

The Antecedents of Poor Doctor-Patient Relationship in Mobile Consultation: A Perspective from Computer-Mediated Communication

This study aims to understand the underlying reasons for poor doctor-patient relationships (DPR). While extant studies on antecedents of poor DPR mainly focus on the offline context and often adopt the patients' perspective, this work focuses on the mobile context and take both doctors' and mobile consultation users' perspectives into consideration. To fulfill this purpose, we first construct a theoretical framework based on the Computer-Mediated Communication (CMC) literature. Then we coded 592 doctor-user communication records to validate and elaborate the proposed theoretical model. This work reveals that characteristics of mobile technologies pose potential challenges on both doctors' and patients' information providing, informative interpreting, and relationship maintaining behaviors, resulting in 10 and 6 types of inappropriate behaviors of doctors and users, respectively, that trigger poor DPR in the mobile context. The findings enrich the research on online DPR and provide insights for improving DPR in the mobile context.

Clinic Satisfaction Tool Improves Communication and Provides Real-Time Feedback

BACKGROUND

Patient-reported assessments of the clinic experience are increasingly important for improving the delivery of care. The Clinician and Group Consumer Assessment of Healthcare Providers and Systems (CG-CAHPS) survey is the current standard for evaluating patients' clinic experience, but its format gives 2-mo delayed feedback on a small proportion of patients in clinic. Furthermore, it fails to give specific actionable results on individual encounters.

OBJECTIVE

To develop and assess the impact of a single-page Clinic Satisfaction Tool (CST) to demonstrate real-time feedback, individualized responses, interpretable and actionable feedback, improved patient satisfaction and communication scores, increased physician buy-in, and overall feasibility.

METHODS

We assessed CST use for 12 mo and compared patient-reported outcomes to the year prior. We assessed all clinic encounters for patient satisfaction, all physicians for CG-CAHPS global rating, and physician communication scores, and evaluated the physician experience 1 yr after implementation.

RESULTS

During implementation, 14 690 patients were seen by 12 physicians, with a 96% overall CST utilization rate. Physicians considered the CST superior to CG-CAHPS in providing immediate feedback. CG-CAHPS global scores trended toward improvement and were predicted by CST satisfaction scores (P < .05). CG-CAHPS physician communication scores were also predicted by CST satisfaction scores (P < .01). High CST satisfaction scores were predicted by high utilization (P < .05). Negative feedback dropped significantly over the course of the study (P < .05).

CONCLUSION

The CST is a low-cost, high-yield improvement to the current method of capturing the clinic experience, improves communication and satisfaction between physicians and patients, and provides real-time feedback to physicians.

Diagnostic route is associated with care satisfaction independently of tumour stage: Evidence from linked English Cancer Patient Experience Survey and cancer registration data

BACKGROUND

Whether diagnostic route (e.g. emergency presentation) is associated with cancer care experience independently of tumour stage is unknown.

METHODS

We analysed data on 18 590 patients with breast, prostate, colon, lung, and rectal cancers who responded to the 2014 English Cancer Patient Experience Survey, linked to cancer registration data on diagnostic route and tumour stage at diagnosis. We estimated odds ratios (OR) of reporting a negative experience of overall cancer care by tumour stage and diagnostic route (crude and adjusted for patient characteristic and cancer site variables) and examined their interactions with cancer site.

RESULTS

After adjustment, the likelihood of reporting a negative experience was highest for emergency presenters and lowest for screening-detected patients with breast, colon, and rectal cancers (OR versus two-week-wait 1.51, 95% confidence interval [CI] 1.24-1.83; 0.88, 95% CI 0.75-1.03, respectively). Patients with the most advanced stage were more likely to report a negative experience (OR stage IV versus I 1.37, 95% CI 1.15-1.62) with little confounding between stage and route, and no evidence for cancer-stage or cancer-route interactions.

CONCLUSIONS

Though the extent of disease is strongly associated with ratings of overall cancer care, diagnostic route (particularly emergency presentation or screening detection) exerts important independent effects.

Culturally-adapted cognitive behavioural therapy based intervention for maternal depression: a mixed-methods feasibility study

BACKGROUND

British Pakistanis are one of the largest ethnic minority groups living in the UK, with high rates of maternal depression being reported in this population. Evidence suggests that culturally-adapted Cognitive Behavioural Therapy (CBT)-based interventions for depression, may improve clinical outcomes and patient satisfaction. This study was conducted to develop and test the feasibility and acceptability of a culturally-adapted, CBT-based, manual-assisted intervention in British Pakistani mothers experiencing maternal depression.

METHODS

A mixed-method feasibility study that included qualitative interviews followed by the development of a CBT-based intervention for mothers with mild to moderate depression. Following the qualitative interviews, a CBT-based intervention called the Positive Health Program (PHP) was developed and delivered consisting of 12-weekly sessions. A before and after design was used to explore the feasibility and acceptability of the Positive Health Programme.

RESULTS

A culturally-adapted CBT-based group intervention (PHP) was acceptable to this group and improvements were reported in depression and health-related quality of life. The women's understanding of 'depression' as a general consensus was in physical terms, but with an onset triggered by psychosocial causes. The most commonly reported factors contributing to depression were marital disharmony, lack of social support, and financial difficulties. Past help offered was primarily antidepressants, which were not welcomed by most of the women. A lack of availability of culturally sensitive interventions and the limited cultural sensitivity of NHS staff was also reported.

CONCLUSION

This study provides preliminary evidence for the feasibility and acceptability of a CBT-based culturally-adapted group psychological intervention for British Pakistani mothers.

TRIAL REGISTRATION

Study ethics registration number: 10/H1005/62 (University of Manchester).

Language matters: a UK perspective

AIM

To review the existing evidence regarding the use of language in clinical encounters.

BACKGROUND

Awareness of the importance of language in clinical encounters is mostly lacking or located within broader discussions on communication.

METHODS

A scoping study was conducted to review existing research that could increase our understanding of the role language plays as well as identify gaps in knowledge and inform the development of a position statement on language in diabetes care.

RESULTS

Evidence shows that, although carefully chosen language can have a positive effect, there is a potential negative impact of language on people's experiences of diabetes care. The use of stigmatizing and discriminatory words during communication between healthcare practitioners and people with diabetes can lead to disengagement with health services as well as sub-optimal diabetes self-management. Clinical encounters can be compromised where language barriers exist or where there is limited understanding of cultural differences that may have an impact on diabetes self-management. What little empirical evidence there is shows that training can improve language and communication skills.

CONCLUSION

This review raises a number of questions that are being addressed by the NHS England Language Matters Group, which has developed a set of recommendations to support the use of appropriate language in clinical encounters.

Unhappy Patients Are Not Alike: Content Analysis of the Negative Comments from China's Good Doctor Website

BACKGROUND

With the rise in popularity of Web 2.0 technologies, the sharing of patient experiences about physicians on online forums and medical websites has become a common practice. However, negative comments posted by patients are considered to be more influential by other patients and physicians than those that are satisfactory.

OBJECTIVE

The aim of this study was to analyze negative comments posted online about physicians and to identify possible solutions to improve patient satisfaction, as well as their relationship with physicians.

METHODS

A Java-based program was developed to collect patient comments on the Good Doctor website, one of the most popular online health communities in China. A total of 3012 negative comments concerning 1029 physicians (mean 2.93 [SD 4.14]) from 5 highly ranked hospitals in Beijing were extracted for content analysis. An initial coding framework was constructed with 2 research assistants involved in the codification.

RESULTS

Analysis, based on the collected 3012 negative comments, revealed that unhappy patients are not alike and that their complaints cover a wide range of issues experienced throughout the whole process of medical consultation. Among them, physicians in Obstetrics and Gynecology (606/3012, 20.12%; P=.001) and Internal Medicine (487/3012, 16.17%; P=.80) received the most negative comments. For negative comments per physician, Dermatology and Sexually Transmitted Diseases (mean 5.72, P<.001) and Andrology (mean 5, P=.02) ranked the highest. Complaints relating to insufficient medical consultation duration (577/3012, 19.16%), physician impatience (527/3012, 17.50%), and perceived poor therapeutic effect (370/3012, 12.28%) received the highest number of negative comments. Specific groups of people, such as those accompanying older patients or children, traveling patients, or very important person registrants, were shown to demonstrate little tolerance for poor medical service.

CONCLUSIONS

Analysis of online patient complaints provides an innovative approach to understand factors associated with patient dissatisfaction. The outcomes of this study could be of benefit to hospitals or physicians seeking to improve their delivery of patient-centered services. Patients are expected to be more understanding of overloaded physicians' workloads, which are impacted by China's stretched medical resources, as efforts are made to build more harmonious physician-patient relationships.

Improving patient experience in primary care: a multimethod programme of research on the measurement and improvement of patient experience

Background

There has been an increased focus towards improving quality of care within the NHS in the last 15 years; as part of this, there has been an emphasis on the importance of patient feedback within policy, through National Service Frameworks and the Quality and Outcomes Framework. The development and administration of large-scale national patient surveys to gather representative data on patient experience, such as the national GP Patient Survey in primary care, has been one such initiative. However, it remains unclear how the survey is used by patients and what impact the data may have on practice.

Objectives

Our research aimed to gain insight into how different patients use surveys to record experiences of general practice; how primary care staff respond to feedback; and how to engage primary care staff in responding to feedback.

Methods

We used methods including quantitative survey analyses, focus groups, interviews, an exploratory trial and an experimental vignette study.

Results

(1)Understanding patient experience data. Patients readily criticised their care when reviewing consultations on video, although they were reluctant to be critical when completing questionnaires. When trained raters judged communication during a consultation to be poor, a substantial proportion of patients rated the doctor as ‘good’ or ‘very good’. Absolute scores on questionnaire surveys should be treated with caution; they may present an overoptimistic view of general practitioner (GP) care. However, relative rankings to identify GPs who are better or poorer at communicating may be acceptable, as long as statistically reliable figures are obtained. Most patients have a particular GP whom they prefer to see; however, up to 40% of people who have such a preference are unable regularly to see the doctor of their choice. Users of out-of-hours care reported worse experiences when the service was run by a commercial provider than when it was run by a not-for profit or NHS provider. (2)Understanding patient experience in minority ethnic groups. Asian respondents to the GP Patient Survey tend to be registered with practices with generally low scores, explaining about half of the difference in the poorer reported experiences of South Asian patients than white British patients. We found no evidence that South Asian patients used response scales differently. When viewing the same consultation in an experimental vignette study, South Asian respondents gave higher scores than white British respondents. This suggests that the low scores given by South Asian respondents in patient experience surveys reflect care that is genuinely worse than that experienced by their white British counterparts. We also found that service users of mixed or Asian ethnicity reported lower scores than white respondents when rating out-of-hours services. (3)Using patient experience data. We found that measuring GP–patient communication at practice level masks variation between how good individual doctors are within a practice. In general practices and in out-of-hours centres, staff were sceptical about the value of patient surveys and their ability to support service reconfiguration and quality improvement. In both settings, surveys were deemed necessary but not sufficient. Staff expressed a preference for free-text comments, as these provided more tangible, actionable data. An exploratory trial of real-time feedback (RTF) found that only 2.5% of consulting patients left feedback using touch screens in the waiting room, although more did so when reminded by staff. The representativeness of responding patients remains to be evaluated. Staff were broadly positive about using RTF, and practices valued the ability to include their own questions. Staff benefited from having a facilitated session and protected time to discuss patient feedback.

Conclusions

Our findings demonstrate the importance of patient experience feedback as a means of informing NHS care, and confirm that surveys are a valuable resource for monitoring national trends in quality of care. However, surveys may be insufficient in themselves to fully capture patient feedback, and in practice GPs rarely used the results of surveys for quality improvement. The impact of patient surveys appears to be limited and effort should be invested in making the results of surveys more meaningful to practice staff. There were several limitations of this programme of research. Practice recruitment for our in-hours studies took place in two broad geographical areas, which may not be fully representative of practices nationally. Our focus was on patient experience in primary care; secondary care settings may face different challenges in implementing quality improvement initiatives driven by patient feedback. Recommendations for future research include consideration of alternative feedback methods to better support patients to identify poor care; investigation into the factors driving poorer experiences of communication in South Asian patient groups; further investigation of how best to deliver patient feedback to clinicians to engage them and to foster quality improvement; and further research to support the development and implementation of interventions aiming to improve care when deficiencies in patient experience of care are identified.

Funding

The National Institute for Health Research Programme Grants for Applied Research programme.