A methodological study to compare survey-based and observation-based evaluations of organisational and safety cultures and then compare both approaches with markers of the quality of care

BACKGROUND

Patient safety concerns have focused attention on organisational and safety cultures, in turn directing attention to the measurement of organisational and safety climates.

OBJECTIVES

First, to compare levels of agreement between survey- and observation-based measures of organisational and safety climates/cultures and to compare both measures with criterion-based audits of the quality of care, using evidence-based markers drawn from national care standards relating to six common clinical conditions. (This required development of an observation-based instrument.) Second, to examine whether observation-based evaluations could replace or augment survey measurements to mitigate concerns about declining response rates and increasing social desirability bias. Third, to examine mediating factors in safety and organisational climate scores.

DESIGN

The study had three strands: (A) a postal questionnaire survey to elicit staff perceptions of organisational and safety climates, using six prevalidated scales; (B) semistructured non-participant observation of clinical teams; and (C) a retrospective criterion-based audit carried out by non-clinical auditors to minimise hindsight bias. Standardised summary scores were created for each strand, and pairs of measurements were compared (strand A with strand B, strand A with strand C, and strand B with strand C) using Bland-Altman plots to evaluate agreement. Correlations were also examined. Multilevel modelling of Strand A scores explored mediating factors.

SETTING AND PARTICIPANTS

Eight emergency departments and eight maternity units in England, UK.

INTERVENTIONS

None.

MAIN OUTCOME MEASURES

Examination of feasibility, correlation and agreement.

RESULTS

Strand A: the overall response rate was 27.6%, whereas site-specific rates ranged from 9% to 47%. We identified more mediating factors than previous studies; variable response rates had little effect on the results. Organisational and safety climate scores were strongly correlated (r = 0.845) and exhibited good agreement [standard deviation (SD) differences 0.449; 14 (88%) within ± 0.5; one large difference]. Two commonly used scales had high levels of positive responses, suggesting positive climates or social desirability bias. Strand B: scoring on a four-point scale was feasible. Observational evaluation of teamwork culture was good but too limited for evaluating organisational culture. Strand C: a total of 359-399 cases were audited per condition. The results varied widely between different markers for the same condition, so selection matters. Each site performed well on some markers but not others, with few consistent patterns. Some national guidelines were contested. Comparisons: the comparison of safety climate (survey) and teamwork culture (observation) revealed a moderately low correlation (r = 0.316) and good agreement [SD differences 1.082; 7 (44%) within ±0.5; one large difference]. The comparison of safety climate (survey) and performance (audit) revealed lower correlation (r = 0.150, i.e. relationship not linear) but reasonably good agreement [SD differences 0.992; 9 (56%) within ± 0.5; two large differences]. Comparisons between performance (audit) and both organisational climate (survey) and teamwork culture (observation) showed negligible correlations (< 0.1) but moderately good agreement [SD differences 1.058 and 1.241; 6 (38%) and 7 (44%) within ± 0.5; each with two large differences (at different sites)]. Field notes illuminated large differences.

CONCLUSIONS

Climate scores from staff surveys are not unduly affected by survey response rates, but increasing use risks social desirability bias. Safety climate provides a partial indicator of performance, but qualitative data are needed to understand discrepant results. Safety climate (surveys) and, to a lesser degree, teamwork culture (observations) are better indicators of performance than organisational climate (surveys) or attempts to evaluate organisational culture from time-limited observations. Scoring unobtrusive, time-limited observations to evaluate teamwork culture is feasible, but the instrument developed for this study needs further testing. A refined observation-based measure would be useful to augment or replace surveys.

FUNDING

The National Institute for Health Research Health Technology Assessment programme.

Modafinil for excessive daytime sleepiness in myotonic dystrophy type 1--the patients' perspective

Excessive daytime sleepiness (EDS), of very similar pattern to that seen in narcolepsy syndrome, is extremely common in myotonic dystrophy type 1 (DM1). In a significant minority it has a profound disabling effect on employment, social functioning and activities of daily living. Limited published studies have shown inconsistent results from use of the psychostimulant drug modafinil. A recent European Medicines Agency (EMA) review concluded that on current evidence regarding safety and efficacy, modafinil's use should be restricted to the treatment of narcolepsy. In other conditions (although DM1 was not specifically considered) it was concluded that there was insufficient evidence of benefit to outweigh potentially serious side-effects, including severe skin reactions and cardiac arrhythmia. Clinicians with extensive experience in the management of DM1 have found modafinil to be extremely effective in appropriately selected patients with a very low incidence of serious side-effects. Given the recent EMA review, patients have expressed concern about the potential restriction of the use of modafinil in DM1. This brief review is an audit of the experience of a large group of patients and their clinicians concerning EDS and DM1 and concludes that despite the limited literature there is strong evidence to support the use of modafinil in carefully selected patients.

Capturing information needs of care providers to support knowledge sharing and distributed decision making

BACKGROUND

This paper reports on work carried out to elicit information needs at a trans-disciplinary, nurse-managed health care clinic that serves a medically disadvantaged urban population. The trans-disciplinary model provides a "one-stop shop" for patients who can receive a wide range of services beyond traditional primary care. However, this model of health care presents knowledge sharing challenges because little is known about how data collected from the non-traditional services can be integrated into the traditional electronic medical record (EMR) and shared with other care providers. There is also little known about how health information technology (HIT) can be used to support the workflow in such a practice.

OBJECTIVES

The objective of this case study was to identify the information needs of care providers in order to inform the design of HIT to support knowledge sharing and distributed decision making.

METHODS

A participatory design approach is presented as a successful technique to specify requirements for HIT applications that can support a trans-disciplinary model of care.

RESULTS

Using this design approach, the researchers identified the information needs of care providers working at the clinic and suggested HIT improvements to integrate non-traditional information into the EMR. These modifications allow knowledge sharing among care providers and support better health decisions.

CONCLUSIONS

We have identified information needs of care providers as they are relevant to the design of health information systems. As new technology is designed and integrated into various workflows it is clear that understanding information needs is crucial to acceptance of that technology.

Advancing Breast Cancer HER2 FISH Quality by Image Analysis

Background: A critically important factor in the correlation of Her2 amplification and clinical outcome in breast cancer is the quality of histologic Her2 amplification detection. FISH is the “gold standard” for HER2 amplification detection but presently most labs use manual methods which are labour intensive and restricted by analysis time to small samples (60 cells or less if amplification status appeared clear to the observer).An ideal FISH detection system should be rapid (less than 30 minutes per sample), identify HER2/neu and CEP17 copy number objectively in each cell, count sufficient cells to be representative statistically, count Her2/neu in cell clusters objectively, and have a permanent record of each cell counted. Our objective is to develop an image analysis system for Her2/neu FISH that is superior to current manual and automated methods in all the above criteria.Materials and Methods: The Visiopharm Integrator System software and a Leica DM6000B fluorescence microscope equipped with a Prior 8-slide capacity motorized stage (Mac 5000 ps system stage control) and Hamamatsu camera (Model CA4742-80-12AG) were used for image analysis. Forty breast cancer biopsies, 15 core biopsies included, were examined for Her2 FISH previously assessed manually on the same slide. For manual detection, at least 3 representative fields were selected by the observer. For image analysis, unbiased tumour sampling, typically 16 fields, was assessed within a region of interest previously identified. Methods were compared by ASCO/CAP amplification criteria and by assessment of technical time, cells counted and objectivity of counting criteria.Results:Comparison manual vs image analysisMethodManualImage AnalysisTechnical time: minutes, average, range50 (45-135)25 (20-40)Cells counted: average, range58 (20-217)238 (18-1151)HER2 in clusterssubjectiveobjective and reproducibleASCO/CAP amplified/equivocal19/217/3ASCO/CAP not amplified/uninterpretable18/120/0 Seven discordant cases were observed. Five cases were downgraded by image analysis. Two cases, one uninterpretable manually and one seen as not amplified manually, were seen as amplified by image analysis.Discussion: Image analysis FISH with Visiopharm software allows for establishment of finite cell inclusion criteria reflecting size, circularity and other measurable cell features. Image analysis facilitates higher cell counts without observer selection bias in less time, and with smaller increases in technical time as more cells are counted. Discordance may be attributable to heterogeneity with larger sample of cells assessed and objective assessment of clusters in the image analysis method. Our image analysis protocol demonstrated successfully the quantification of HER2 in separate signals, in clusters and in split signals. HER2/CEP17 copy numbers were determined for each cell and for more cells in much less time while providing a permanent image record of all cells assessed.Image analysis has promise to improve substantially the quality of Her2 FISH assessment in breast cancer biopsies.Supported in part by an unrestricted grant from Hoffman-LaRoche Ltd.

Citation Information: Cancer Res 2009;69(24 Suppl):Abstract nr 6015.

Farnesyl diphosphate synthase is involved in the resistance to zoledronic acid of osteosarcoma cells

We recently demonstrated original anti-tumor effects of zoledronic acid (Zol) on osteosarcoma cell lines independently of their p53 and Rb status. The present study investigated the potential Zol-resistance acquired by osteosarcoma cells after prolonged treatment. After 12 weeks of culture in the presence of 1 μm Zol, the effects of high doses of Zol (10–100 μm) were compared between the untreated rat (OSRGA, ROS) and human (MG63, SAOS2) osteosarcoma cells and Zol-pretreated cells in terms of cell proliferation, cell cycle analysis, migration assay and cytoskeleton organization. Long-term treatment with 1 μm Zol reduced the sensitivity of osteosarcoma cells to high concentrations of Zol. Furthermore, the Zol-resistant cells were sensitive to conventional anti-cancer agents demonstrating that this resistance process is independent of the multidrug resistance phenotype. However, as similar experiments performed in the presence of clodronate and pamidronate evidenced that this drug resistance was restricted to the nitrogen-containing bisphosphonates, we then hypothesized that this resistance could be associated with a differential expression of farnesyl diphos-phate synthase (FPPS) also observed in human osteosarcoma samples. The transfection of Zol-resistant cells with FPPS siRNA strongly increased their sensitivity to Zol. This study demonstrates for the first time the induction of metabolic resistance after prolonged Zol treatment of osteosarcoma cells confirming the therapeutic potential of Zol for the treatment of bone malignant pathologies, but points out the importance of the treatment regimen may be important in terms of duration and dose to avoid the development of drug metabolic resistance.

Provider-initiated HIV testing and counseling in TB clinical settings: tools for program implementation

In countries with high HIV prevalence, up to 80% of tuberculosis (TB) patients also have human immunodeficiency virus (HIV) infection. HIV testing and counseling needs to be more feasible and accessible to all people in settings with a generalized HIV epidemic, but particularly to those likely to have HIV infection and who need care and treatment. Implementing provider-initiated and -delivered HIV testing and counseling (PITC) in clinical settings where patients have symptoms and signs consistent with HIV-related disease, including TB, is therefore a priority. We describe a new tool that has been developed to assist countries in planning and implementing PITC in TB clinical settings. The materials include a template for national guidance for the PITC program and procedures, a training curriculum for clinic staff and job aids including a script that assist clinicians in communicating appropriate pre- and post-test information to their TB patients, including the benefits to HIV-infected TB patients of knowing their status so they can obtain HIV care and treatment and prevent the spread of HIV.