What We Do, What We Say, What We Don't Say: Confidentiality In The Publication Of Clinical Writing

Questions concerning analysts' publication of material from the analyses of their patients have troubled the field of psychoanalysis since its inception. Disguise inevitably distorts the clinical material and is often insufficient to protect the patient from recognition. Asking the patient's consent for publication intrudes upon and alters the analytic process. While analysts have largely reached a consensus about the need for anonymity in published material, there is still considerable debate about the necessity for obtaining patients' consent when using their material for publication. In this paper, I will trace the evolving meanings of disguise, and particularly of consent, in the analytic literature. I will place a particular emphasis upon the differing theoretical belief systems that underlie the analyst's decision to ask consent from her patient or not to do so, and I will argue that, although decisions on asking consent remain a complex matter, such coherent belief systems should play an important part in analysts' decisions regarding consent. I will illustrate my thought processes and some clinical situations with brief examples, and I will conclude with some practical recommendations, with the hope that these will stimulate further discussion in the analytic community.

A prospective survey on therapeutic inertia in psoriatic arthritis (OPTI'PsA)

OBJECTIVES

Clinical inertia, or therapeutic inertia (TI), is the medical behaviour of not initiating or intensifying treatment when recommended by clinical recommendations. To our knowledge, our survey is the first to assess TI around psoriatic arthritis (PsA).

METHODS

Eight hundred and twenty-five French rheumatologists were contacted via email between January and March 2021 and invited to complete an online questionnaire consisting of seven clinical vignettes: five cases ('oligoarthritis', 'enthesitis', 'polyarthritis', 'neoplastic history', 'cardiovascular risk') requiring treatment OPTImization, and two 'control' cases (distal interphalangeal arthritis, atypical axial involvement) not requiring any change of treatment-according to the most recent PsA recommendations. Rheumatologists were also questioned about their routine practice, continuing medical education and perception of PsA.

RESULTS

One hundred and one rheumatologists completed this OPTI'PsA survey. Almost half the respondents (47%) demonstrated TI on at least one of the five vignettes that warranted treatment optimization. The complex profiles inducing the most TI were 'oligoarthritis' and 'enthesitis' with 20% and 19% of respondents not modifying treatment, respectively. Conversely, clinical profiles for which there was the least uncertainty ('polyarthritis in relapse', 'neoplastic history' and 'cardiovascular risk') generated less TI with 11%, 8% and 6% of respondents, respectively, choosing not to change the current treatment.

CONCLUSION

The rate of TI we observed for PsA is similar to published data for other chronic diseases such as diabetes, hypertension, gout or multiple sclerosis. Our study is the first to show marked clinical inertia in PsA, and further research is warranted to ascertain the reasons behind this inertia.

Assessment of a Digital Symptom Checker Tool's Accuracy in Suggesting Reproductive Health Conditions: Clinical Vignettes Study

BACKGROUND

Reproductive health conditions such as endometriosis, uterine fibroids, and polycystic ovary syndrome (PCOS) affect a large proportion of women and people who menstruate worldwide. Prevalence estimates for these conditions range from 5% to 40% of women of reproductive age. Long diagnostic delays, up to 12 years, are common and contribute to health complications and increased health care costs. Symptom checker apps provide users with information and tools to better understand their symptoms and thus have the potential to reduce the time to diagnosis for reproductive health conditions.

OBJECTIVE

This study aimed to evaluate the agreement between clinicians and 3 symptom checkers (developed by Flo Health UK Limited) in assessing symptoms of endometriosis, uterine fibroids, and PCOS using vignettes. We also aimed to present a robust example of vignette case creation, review, and classification in the context of predeployment testing and validation of digital health symptom checker tools.

METHODS

Independent general practitioners were recruited to create clinical case vignettes of simulated users for the purpose of testing each condition symptom checker; vignettes created for each condition contained a mixture of condition-positive and condition-negative outcomes. A second panel of general practitioners then reviewed, approved, and modified (if necessary) each vignette. A third group of general practitioners reviewed each vignette case and designated a final classification. Vignettes were then entered into the symptom checkers by a fourth, different group of general practitioners. The outcomes of each symptom checker were then compared with the final classification of each vignette to produce accuracy metrics including percent agreement, sensitivity, specificity, positive predictive value, and negative predictive value.

RESULTS

A total of 24 cases were created per condition. Overall, exact matches between the vignette general practitioner classification and the symptom checker outcome were 83% (n=20) for endometriosis, 83% (n=20) for uterine fibroids, and 88% (n=21) for PCOS. For each symptom checker, sensitivity was reported as 81.8% for endometriosis, 84.6% for uterine fibroids, and 100% for PCOS; specificity was reported as 84.6% for endometriosis, 81.8% for uterine fibroids, and 75% for PCOS; positive predictive value was reported as 81.8% for endometriosis, 84.6% for uterine fibroids, 80% for PCOS; and negative predictive value was reported as 84.6% for endometriosis, 81.8% for uterine fibroids, and 100% for PCOS.

CONCLUSIONS

The single-condition symptom checkers have high levels of agreement with general practitioner classification for endometriosis, uterine fibroids, and PCOS. Given long delays in diagnosis for many reproductive health conditions, which lead to increased medical costs and potential health complications for individuals and health care providers, innovative health apps and symptom checkers hold the potential to improve care pathways.

The Accuracy and Potential Racial and Ethnic Biases of GPT-4 in the Diagnosis and Triage of Health Conditions: Evaluation Study

BACKGROUND

Whether GPT-4, the conversational artificial intelligence, can accurately diagnose and triage health conditions and whether it presents racial and ethnic biases in its decisions remain unclear.

OBJECTIVE

We aim to assess the accuracy of GPT-4 in the diagnosis and triage of health conditions and whether its performance varies by patient race and ethnicity.

METHODS

We compared the performance of GPT-4 and physicians, using 45 typical clinical vignettes, each with a correct diagnosis and triage level, in February and March 2023. For each of the 45 clinical vignettes, GPT-4 and 3 board-certified physicians provided the most likely primary diagnosis and triage level (emergency, nonemergency, or self-care). Independent reviewers evaluated the diagnoses as "correct" or "incorrect." Physician diagnosis was defined as the consensus of the 3 physicians. We evaluated whether the performance of GPT-4 varies by patient race and ethnicity, by adding the information on patient race and ethnicity to the clinical vignettes.

RESULTS

The accuracy of diagnosis was comparable between GPT-4 and physicians (the percentage of correct diagnosis was 97.8% (44/45; 95% CI 88.2%-99.9%) for GPT-4 and 91.1% (41/45; 95% CI 78.8%-97.5%) for physicians; P=.38). GPT-4 provided appropriate reasoning for 97.8% (44/45) of the vignettes. The appropriateness of triage was comparable between GPT-4 and physicians (GPT-4: 30/45, 66.7%; 95% CI 51.0%-80.0%; physicians: 30/45, 66.7%; 95% CI 51.0%-80.0%; P=.99). The performance of GPT-4 in diagnosing health conditions did not vary among different races and ethnicities (Black, White, Asian, and Hispanic), with an accuracy of 100% (95% CI 78.2%-100%). P values, compared to the GPT-4 output without incorporating race and ethnicity information, were all .99. The accuracy of triage was not significantly different even if patients' race and ethnicity information was added. The accuracy of triage was 62.2% (95% CI 46.5%-76.2%; P=.50) for Black patients; 66.7% (95% CI 51.0%-80.0%; P=.99) for White patients; 66.7% (95% CI 51.0%-80.0%; P=.99) for Asian patients, and 62.2% (95% CI 46.5%-76.2%; P=.69) for Hispanic patients. P values were calculated by comparing the outputs with and without conditioning on race and ethnicity.

CONCLUSIONS

GPT-4's ability to diagnose and triage typical clinical vignettes was comparable to that of board-certified physicians. The performance of GPT-4 did not vary by patient race and ethnicity. These findings should be informative for health systems looking to introduce conversational artificial intelligence to improve the efficiency of patient diagnosis and triage.

Assessing the Utility of ChatGPT Throughout the Entire Clinical Workflow: Development and Usability Study

BACKGROUND

Large language model (LLM)-based artificial intelligence chatbots direct the power of large training data sets toward successive, related tasks as opposed to single-ask tasks, for which artificial intelligence already achieves impressive performance. The capacity of LLMs to assist in the full scope of iterative clinical reasoning via successive prompting, in effect acting as artificial physicians, has not yet been evaluated.

OBJECTIVE

This study aimed to evaluate ChatGPT's capacity for ongoing clinical decision support via its performance on standardized clinical vignettes.

METHODS

We inputted all 36 published clinical vignettes from the Merck Sharpe & Dohme (MSD) Clinical Manual into ChatGPT and compared its accuracy on differential diagnoses, diagnostic testing, final diagnosis, and management based on patient age, gender, and case acuity. Accuracy was measured by the proportion of correct responses to the questions posed within the clinical vignettes tested, as calculated by human scorers. We further conducted linear regression to assess the contributing factors toward ChatGPT's performance on clinical tasks.

RESULTS

ChatGPT achieved an overall accuracy of 71.7% (95% CI 69.3%-74.1%) across all 36 clinical vignettes. The LLM demonstrated the highest performance in making a final diagnosis with an accuracy of 76.9% (95% CI 67.8%-86.1%) and the lowest performance in generating an initial differential diagnosis with an accuracy of 60.3% (95% CI 54.2%-66.6%). Compared to answering questions about general medical knowledge, ChatGPT demonstrated inferior performance on differential diagnosis (β=-15.8%; P<.001) and clinical management (β=-7.4%; P=.02) question types.

CONCLUSIONS

ChatGPT achieves impressive accuracy in clinical decision-making, with increasing strength as it gains more clinical information at its disposal. In particular, ChatGPT demonstrates the greatest accuracy in tasks of final diagnosis as compared to initial diagnosis. Limitations include possible model hallucinations and the unclear composition of ChatGPT's training data set.

Initiative to Improve Evidence-Based Chronic Obstructive Pulmonary Disease Hospitalist Care Using a Novel On-Line Gamification Patient Simulation Tool: A Prospective Study

Chronic obstructive pulmonary disease (COPD) remains a leading cause of morbidity and mortality. Much of the disease burden comes from exacerbations requiring hospitalization. Unwarranted care variation and divergence from evidence-based COPD management guidelines among hospitalists is a leading driver of the poor outcomes and excess costs associated with COPD-related hospitalizations. We engaged with Novant Health hospitalists to determine if measurement and feedback using fixed-choice simulated patients improves evidence-based care delivery and reduces costs. We created a series of gamified acute-care COPD case simulations with real-time feedback over 16 weeks then performed a year-over-year analytic comparison of the cost, length of stay (LOS), and revisits over the six months prior to the introduction of the simulated patients, the four months while caring for the simulated patients, and the six months after. In total, 245 hospitalists from 15 facilities at Novant Health participated. At baseline, the overall quality-of-care was measured as 58.4% + 12.3%, with providers correctly identifying COPD exacerbation in 92.4% of cases but only identifying the grade and group in 61.9% and 49.5% of cases, respectively. By the study end, the quality-of-care had improved 10.5% (p < 0.001), including improvements in identifying the grade (+9.7%, p = 0.044) and group (+8.4%, p = 0.098). These improvements correlated with changes in real-world performance data, including a 19% reduction in COPD-related pharmacy costs. Overall, the annualized impact of COPD improvements led to 233 fewer inpatient days, 371 fewer revisit days, and inpatient savings totaling nearly $1 million. Engaging practicing providers with patient simulation-based serial measurements and gamified evidence-based feedback potentially reduces inpatient costs while simultaneously reducing patient LOS and revisit rates.

Assessing provider performance of intrapartum care using simulated encounters and clinical vignettes: A comparison study from Tanzania

Objective

To compare clinical vignettes and objective structured clinical examinations (OSCE) as methods for assessing the quality of intrapartum care among skilled providers in rural primary‐level health facilities in Tanzania.

Methods

Cross‐sectional study conducted at six health facilities in the Simiyu region of Tanzania. Providers were assessed using OSCE and clinical vignettes in spontaneous delivery, neonatal resuscitation, and management of postpartum hemorrhage. Trained researchers used a structured clinical checklist. The frequencies of items are presented as percentages and the agreement of the methods of assessment are reported using kappa statistics (high: kappa > 0.80, moderate: kappa = 0.60–0.80, low: kappa < 0.60).

Results

Most healthcare providers were female (60.7%), registered nurses by training (29.0%), and worked in a dispensary (56.1%), with an average age of 33 years and an average of 7.4 years of experience in their respective professions. Five items had high agreement between OSCE and clinical vignettes: postpartum vital signs every 15 min, oxytocin within 1 min of birth, diagnosis of postpartum hemorrhage, elevating legs of the mother, and deciding on manual compression of the uterus.

Conclusion

OSCE and clinical vignettes should be viewed as complimentary to one another in the assessment of provider knowledge and skill, with priority given to OSCE, particularly in intrapartum care.

Clinical vignettes and objective structured clinical examinations are complimentary methods of assessment of intrapartum care in Tanzania, with priority given to objective structured clinical examinations.

General practitioners' clinical practice on the management of cystitis in Normandy, France: A clinical vignettes-based study

RATIONALE, AIMS, AND OBJECTIVES

The objective was to measure the quality of clinical practice for the management of cystitis in adult women in general practice by collaborating with quality circles and the regional centre for antibiotic counsel.

METHOD

This descriptive cross-sectional study was performed in 2018 in Normandy, France. A questionnaire composed of clinical vignettes was used to evaluate practices of general practitioners (GPs) with regard to cystitis classified into four categories: simple, at risk of complication, recurrent, and caused by multidrug-resistant bacteria. The 2017 French Infectious Diseases Society's guidelines were used as a reference.

RESULTS

A total of 142 GPs participated in the study (45.5% of the solicited). Fosfomycin-trometamol and pivmecillinam were cited as first-line treatments for simple cystitis by 134 (94%) and 38 (27%) participants, respectively. For at risk of complication cystitis, the treatments cited were cefixime by 64 participants (45%), ofloxacin by 50 (35%), pivmecillinam by 49 (35%), fosfomycin-trometamol by 38 (27%), nitrofurantoin by 36 (25%), and amoxicillin-clavulanic acid by 28 (20%). Mean compliance rates were 85% for simple cystitis, 39% for at risk of complication cystitis, 60% for recurrent cystitis and 14% for cystitis caused by multidrug-resistant bacteria. Two criteria had less than 10% of the compliant answers: comprehensive knowledge of cystitis complication risk factors (9%) and positivity thresholds of urine cultures (10%).

CONCLUSIONS

In this study, diagnostic means, follow-up testing, and simple cystitis treatment (with fosfomycin predominantly mentioned) were broadly compliant. The use of critical antibiotics was too frequent for at risk of complication cystitis. There may be a need to improve the knowledge of professionals on antibiotic resistance and appropriate antibiotic use.

Examination of physician factors influencing psychiatric assessment of acutely suicidal patients

Suicide risk assessment is a subjective process and remains a clinical challenge in psychiatry. We aimed to examine physicians' characteristics that influence management of acutely suicidal patients. In a cross-sectional design, we performed an anonymous internet survey of psychiatry residents and attendings from four academic centers. Gender, years of experience, practice setting, prior patient suicide, and personal exposure to suicide were characterized. Participants were presented with three clinical vignettes and asked to rate suicide risk and clinical disposition. The relationship between responses to the vignettes and physician characteristics were examined with generalized linear models. Fifty-four residents and 49 attendings completed the survey. Four (7%) residents and 24 (49%) attendings had patients die by suicide, whereas 32 (59%) and 36 (74%), respectively, knew somebody outside their practice who died by suicide. Among residents, lower rating of acute suicide risk was associated with prior exposure to non-patient suicide. Less hospitalization chosen by attendings was associated with greater perceived difficulty of suicide risk assessment. In the combined resident and attending sample, less proneness to hospitalize was associated with number of previous patients die by suicide and with outpatient practice. Our results suggest that previous exposure to suicide is associated with more risk-averse management.

Criminal Responsibility Scale: Development and Validation of a Psychometric Tool Structured in Clinical Vignettes for Criminal Responsibility Assessments in Brazil

Criminal responsibility assessment is undertaken by psychologists or psychiatrists to assess offenders' legal capacities, which vary among countries or regional legislations. There are two psychometric tools (i.e., checklists) validated for criminal responsibility assessment: the Roger Criminal Responsibility Scale, and the rating scale of criminal responsibility for mentally disordered offenders. Despite the existence of psychometric tools structured in clinical vignettes for evaluating legal capacities, none serve the purpose of assessing criminal responsibility. This study aims to validate a novel psychometric tool structured in vignettes for the assessment of criminal responsibility called the "Criminal Responsibility Scale." We applied the tool to 88 defendants referred for criminal responsibility assessment in a forensic medical institute in the city of Rio de Janeiro, Brazil, from December 2017 to December 2018. The validity of the Criminal Responsibility Scale and subscales were evaluated using confirmatory factor analysis. The two-factor solution proved satisfactory and met the needs for practical application of the tool (Kaiser-Meyer-Oklin = 0.82; p < 0.001). Moreover, the inter-rater reliability was evaluated by comparing the tool's final score with that of the expert's conclusion in each case and was found to be satisfactory (k = 0.667-1.0), with a resulting cutoff point of 30.50 (±2) and a Youden index of 0.509. Hence, the Criminal Responsibility Scale is an effective psychometric tool for assessments of criminal responsibility that may encourage future research in assessments of legal capacity with clinical vignette-based psychometric instruments.

Evaluation of a prescription support-tool for chronic management of oral antithrombotic combinations in adults using clinical vignettes: protocol of a randomised controlled trial

INTRODUCTION

Improving the appropriateness of prescriptions of oral antithrombotic (AT) drugs, especially AT combinations, is crucial because these drugs are implicated in bleeding events. We developed a prescription support-tool synthesising guidelines on chronic management of oral AT combinations. Our main objective is to assess the impact of this tool on improving the prescription of oral ATs to comply with guidelines.

METHODS AND ANALYSIS

A randomised controlled trial will be conducted among French general practitioners and cardiologists involved in outpatient settings. Physicians will be invited to participate to an online survey by email via physician associations, social networks or word of mouth. They will be randomised to two arms: the experimental arm (access to the prescription support-tool) or the control arm (no prescription support-tool). Then, all participants will be presented three different clinical vignettes illustrating outpatient clinical situations and will be asked to propose prescriptions for each vignette (number of ATs, type, dosage and duration). A computer-generated randomisation scheme implemented in the online survey will be used to allocate physicians to the experimental or control arm and then stratified by medical specialty. The primary outcome will be fully appropriate prescription of oral ATs ie, that comply with the guidelines in terms of number of drugs, drug class, dosage and duration. To demonstrate a 5% increase in this proportion, we will need to include a minimum of 230 physicians per arm. A logistic mixed model with a clinical vignette-effect and a physician-effect nested in the arm of the study will be used.

ETHICS AND DISSEMINATION

The Institutional Review Board of Inserm (IRB00003888) approved our research project (no. 18-492). If the prescription support-tool improves the prescription of oral ATs, we will create an interactive web tool and will assess its impact in terms of clinical outcomes in real-life.

TRIAL REGISTRATION NUMBER

NCT03630874; Pre-results.

The Utility of the Frailty Index in Clinical Decision Making

Using clinical vignettes, this study aimed to determine if a measure of patient frailty would impact management decisions made by geriatricians regarding commonly encountered clinical situations. Electronic surveys consisting of three vignettes derived from cases commonly seen in an acute inpatient ward were distributed to geriatricians. Vignettes included patients being considered for intensive care treatment, rehabilitation, or coronary artery bypass surgery. A frailty index was generated through Comprehensive electronic Geriatric Assessment. For each vignette, respondents were asked to make a recommendation for management, based on either a brief or detailed amount of clinical information and to reconsider their decision after the addition of the frailty index. The study suggests that quantification of frailty might aid the clinical judgment now employed daily to proceed with usual care, or to modify it based on the vulnerability of the person to whom it is aimed.

Are physicians and clinical pharmacists aware and knowledgeable enough about inappropriate prescribing for elderly patients? Findings from Malaysia

OBJECTIVES

To assess the knowledge of physicians and clinical pharmacists about inappropriate prescribing for elderly patients, their confidence in prescribing for elderly patients, and their perceptions of barriers to appropriate prescribing in this population.

METHODS

A cross-sectional study using a validated 20-item questionnaire was conducted among physicians (n=78) and clinical pharmacists (n=45) working in the medical wards of two tertiary hospitals in Malaysia. Knowledge was assessed by six clinical vignettes which were developed based on Beers criteria and the STOPP/START criteria. Other domains of the study were investigated using a four-point or five-point Likert scale.

RESULTS

Of the 82 participants who completed the questionnaire, 65% were physicians, 90.2% had never received training in geriatric medicine, and 70.8% estimated that 25% or more of their patients were elderly. Only six participants (7.3%) had ever used STOPP/START or Beers criteria when prescribing for elderly patients, and 60% of the respondents had never heard of either one of those criteria. The mean score (SD) for the knowledge part was 3.65 (1.46) points, and only 27 participants (22.9%) scored more than four out of a possible six points. Overall, 34% of the participants rated themselves as confident in prescribing for elderly patients, and this was significantly associated with their knowledge score (P=0.02). The mean number (SD) of barriers cited per participant was 6.88 (2.84), with polypharmacy being the most cited barrier.

CONCLUSIONS

The majority of the participants had inadequate knowledge and low confidence regarding recommending medications for elderly patients. Continuing education on geriatric pharmacotherapy may be of value for the hospital physicians and pharmacists.

Resident Vignettes for Assessing Care Quality in Nursing Homes

OBJECTIVES

Validated process measures that correlate with patient outcomes are needed for research and quality improvement.

DESIGN

Cross-sectional analysis within a cluster-randomized fall prevention study.

SETTING

Nursing homes in North Carolina (n = 16).

PARTICIPANTS

Nursing home staff (n = 541) and residents with 1 or more falls in 6 months (n = 597).

MEASUREMENTS

Fall-prevention process measures in 4 categories derived from Assessing Care of Vulnerable Elders quality indicators were measured in 2 ways: (1) chart abstraction; and (2) staff responses to clinical vignettes of hypothetical residents at risk for falls. Recurrent fall rates (falls/resident/year) were measured. The proportion of the total variation in falls rates explained by the scores for each method (chart abstraction or vignette) was calculated using multilevel adjusted models.

RESULTS

Chart and vignette measures of comorbidity management were moderately correlated (Pearson correlation coefficient 0.43), whereas other process measure categories had low or negative correlation between the 2 methods (psychoactive medication reduction 0.13, environmental modification -0.42, and exercise/rehabilitation -0.08). Measures of environmental modification and comorbidity management explained a moderate amount of the total variation in recurrent fall fates, vignettes (7%-10% variation explained) were superior to chart abstraction (2%-6% variation explained). Vignette responses from unlicensed staff (nurse aides and rehabilitation aides) explained more variance than registered nurses, licensed practical nurses, or other licensed staff in these categories. Process measures for psychoactive medication reduction and exercise/rehabilitation did not explain any of the variation in fall outcomes. Overall, vignette process measures explained 3.9% and chart abstraction measures explained 0% of the variation in fall outcomes.

CONCLUSIONS

Clinical vignettes completed by nursing home staff had greater association with resident recurrent fall rates than traditional chart abstraction process measures.

Study of Individualization and Bias in Nursing Home Fall Prevention Practices

OBJECTIVES

Little is known about how nursing home staff use resident characteristics to individualize care delivery or whether care is affected by implicit bias.

DESIGN

Randomized factorial clinical vignette survey.

SETTING

Sixteen nursing homes in North Carolina.

PARTICIPANTS

Nursing, rehabilitation, and social services staff (n = 433).

MEASUREMENTS

Vignettes describing hypothetical residents were generated from a matrix of clinical and demographic characteristics. Resident age, race and gender were suggested by a photo. Participants completed up to four randomly assigned vignettes (n = 1615), rating the likelihood that 12 fall prevention activities would be used for the resident. Fixed and random effects mixed model analysis examined the impact of vignette resident characteristics and staff characteristics on four intervention categories.

RESULTS

Staff reported a higher likelihood of fall prevention activities in all four categories for residents with a prior fall (0.2-0.5 points higher, 10 point scale, P < 0.05), but other risk factors did not affect scores. There was little evidence of individualization; only dementia increased the reported likelihood of environmental modification (0.3, P < 0.001, 95% CI 0.2-0.5). Individualization did not vary with staff licensure category or clinical experience. Registered nurses consistently reported higher likelihoods of all fall prevention activities than did licensed practical nurses, unlicensed staff and other professional staff (1.0-2.7 points, P < 0.001 to 0.005). There was a small degree of implicit racial bias; staff indicated that environmental modification would be less likely to occur in otherwise identical vignettes including a photo of a black rather than a white resident (-0.2 points, 95% CI -0.3 to -0.1).

CONCLUSION

Nursing home staff report a standardized approach to fall prevention without individualization. We found a small impact from implicit racial bias that should be further explored.

Methods of Observing Variations in Physicians' Decisions: The Opportunities of Clinical Vignettes

To support their efforts to promote high quality and efficient care, policymakers need to better understand the key factors associated with variations in physicians' decisions, and in particular, physician deviations from evidence-based care. Clinical vignette survey instruments hold potential for research in this area as an approach that both allows for practical, large-scale study and overcomes the data quality challenges posed by analysis of clinical data. These surveys present respondents with a narrative description of a hypothetical patient case and solicit responses to one or more questions regarding the care of the patient. In this review, we describe various methods for measuring variations in physicians' decisions and highlight a range of design features researchers should consider when developing a clinical vignette survey. We conclude by identifying areas for future research.

Comparison of clinical vignettes and standardized patients as measures of physiotherapists' activity and work recommendations in patients with non-specific low back pain

OBJECTIVE

To validate clinical vignettes as a measure of physiotherapists' activity and work recommendations given to patients with non-specific low back pain.

DESIGN

Validation study comparing two methods for measuring aspects of health providers' clinical management: Clinical vignettes and unannounced visits of standardized patients (the gold standard).

SETTING

Outpatient physiotherapy clinics.

SUBJECTS

Physiotherapists (N = 59) who consented to see unannounced standardized patients in their clinical practice.

MAIN MEASURES

Clinical vignettes were used to initially measure physiotherapists' self-reported activity and work recommendations. Subsequently, actors performing as standardized patients visited physiotherapists in their clinical practice and rated the advice given by the physiotherapist regarding activity and work. A total of 23 standardized patients were randomly scheduled to physiotherapists. Physiotherapists were blinded towards the standardized patients. To test whether standardized patients were detected, physiotherapists reported if they suspected that they had treated an actor.

RESULTS

The 23 standardized patients visited 22 different physiotherapists. Physiotherapists detected 12 out of 23 unannounced standardized patients (detection rate: 52%). The estimated agreement between the two measures was poor, for both activity and work recommendations (weighted kappa coefficients: 0.29 resp. -0.21).

CONCLUSION

The poor concordance between clinical vignettes and standardized patients indicates the potentially limited validity of clinical vignettes as a measure of health providers' activity and work recommendations in low back pain practice.

EULAR Sjogren's syndrome disease activity index: development of a consensus systemic disease activity index for primary Sjogren's syndrome

OBJECTIVE

To develop a disease activity index for patients with primary Sjögren's syndrome (SS): the European League Against Rheumatism (EULAR) Sjögren's syndrome disease activity index (ESSDAI).

METHODS

Thirty-nine SS experts participated in an international collaboration, promoted by EULAR, to develop the ESSDAI. Experts identified 12 organ-specific 'domains' contributing to disease activity. For each domain, features of disease activity were classified in three or four levels according to their severity. Data abstracted from 96 patients with systemic complications of primary SS were used to generate 702 realistic vignettes for which all possible systemic complications were represented. Using the 0-10 physician global assessment (PhGA) scale, each expert scored the disease activity of five patient profiles and 20 realistic vignettes. Multiple regression modelling, with PhGA used as the dependent variable, was used to estimate the weight of each domain.

RESULTS

All 12 domains were significantly associated with disease activity in the multivariate model, domain weights ranged from 1 to 6. The ESSDAI scores varied from 2 to 47 and were significantly correlated with PhGA for both real patient profiles and realistic vignettes (r=0.61 and r=0.58, respectively, p<0.001). Compared with 57 (59.4%) of the real patient profiles, 468 (66.7%) of the realistic vignettes were considered likely or very likely to be true.

CONCLUSIONS

The ESSDAI is a clinical index designed to measure disease activity in patients with primary SS. Once validated, such a standardised evaluation of primary SS should facilitate clinical research and be helpful as an outcome measure in clinical trials.

Using vignettes to compare the quality of clinical care variation in economically divergent countries

OBJECTIVE

To determine whether clinical vignettes can measure variations in the quality of clinical care in two economically divergent countries.

DATA SOURCE/STUDY SETTING

Primary data collected between February 1997 and February 1998 at two Veterans Affairs facilities in the United States and four government-run outpatient facilities in Macedonia.

STUDY DESIGN

Randomly selected, eligible Macedonian and U.S. physicians (>97 percent participation rate) completed vignettes for four common outpatient conditions. Responses were judged against a master list of explicit quality criteria and scored as percent correct. DATA COLLECTION/ EXTRACTION: An ANOVA model and two-tailed t-tests were used to compare overall scores by case, study site, and country. Principal Findings. The mean score for U.S. physicians was 67 percent (+/-11 percent) compared to 48 percent (+/-11 percent) for Macedonian physicians. The quality of clinical practice, which emphasizes basic skills, varied greatly in both sites, but more so in Macedonia. However, the top Macedonian physicians in all sites approached or-in one case-exceeded the median score in the U.S. sites.

CONCLUSIONS

Vignettes are a useful method for making cross-national comparisons of the quality of care provided in very different settings. The vignette measurements revealed that some physicians in Macedonia performed at a standard comparable to that of their counterparts in the United States, despite the disparity of the two health systems. We infer that in poorer countries, policy that promotes improvements in the quality of clinical practice-not just structural inputs-could lead to rapid improvements in health.

An evaluation of vignettes for predicting variation in the quality of preventive care

OBJECTIVE

Clinical vignettes offer an inexpensive and convenient alternative to the benchmark method of chart audits for assessing quality of care. We examined whether vignettes accurately measure and predict variation in the quality of preventive care.

DESIGN

We developed scoring criteria based on national guidelines for 11 prevention items, categorized as vaccine, vascular-related, cancer screening, and personal behaviors. Three measurement methods were used to ascertain the quality of care provided by clinicians seeing trained actors (standardized patients; SPs) presenting with common outpatient conditions: 1) the abstracted medical record from an SP visit; 2) SP reports of physician practice during those visits; and 3) physician responses to matching computerized case scenarios (clinical vignettes).

SETTING

Three university-affiliated (including 2 VA) and one community general internal medicine clinics.

PATIENTS/PARTICIPANTS

Seventy-one randomly selected physicians from among eligible general internal medicine residents and attending physicians.

MEASUREMENTS AND MAIN RESULTS

Physicians saw 480 SPs (120 at each site) and completed 480 vignettes. We calculated the proportion of prevention items for each visit reported or recorded by the 3 measurement methods. We developed a multiple regression model to determine whether site, training level, or clinical condition predicted prevention performance for each measurement method. We found that overall prevention scores ranged from 57% (SP) to 54% (vignettes) to 46% (chart abstraction). Vignettes matched or exceeded SP scores for 3 prevention categories (vaccine, vascular-related, and personal behavior). Prevention quality varied by site (from 40% to 67%) and was predicted similarly by vignettes and SPs.

CONCLUSIONS

Vignettes can measure and predict prevention performance. Vignettes may be a less costly way to assess prevention performance that also controls for patient case-mix.