Analysis of data over time: a difficult statistical issue

The evaluation of complex interventions in palliative care: an exploration of the potential of case study research strategies

BACKGROUND

Complex, incrementally changing, context dependent and variable palliative care services are difficult to evaluate. Case study research strategies may have potential to contribute to evaluating such complex interventions, and to develop this field of evaluation research.

AIM

This paper explores definitions of case study (as a unit of study, a process, and a product) and examines the features of case study research strategies which are thought to confer benefits for the evaluation of complex interventions in palliative care settings.

RESULTS

Ten features of case study that are thought to be beneficial in evaluating complex interventions in palliative care are discussed, drawing from exemplars of research in this field. Important features are related to a longitudinal approach, triangulation, purposive instance selection, comprehensive approach, multiple data sources, flexibility, concurrent data collection and analysis, search for proving-disproving evidence, pattern matching techniques and an engaging narrative. The limitations of case study approaches are discussed including the potential for subjectivity and their complex, time consuming and potentially expensive nature.

CONCLUSIONS

Case study research strategies have great potential in evaluating complex interventions in palliative care settings. Three key features need to be exploited to develop this field: case selection, longitudinal designs, and the use of rival hypotheses. In particular, case study should be used in situations where there is interplay and interdependency between the intervention and its context, such that it is difficult to define or find relevant comparisons.

Evaluating Disease Management Program Effectiveness: An Introduction to Time-Series Analysis

Currently, the most widely used method in the disease management (DM) industry for evaluating program effectiveness is referred to as the "total population approach." This model is a pretest-posttest design, with the most basic limitation being that without a control group, there may be sources of bias and/or competing extraneous confounding factors that offer a plausible rationale explaining the change from baseline. Furthermore, with the current inclination of DM programs to use financial indicators rather than program-specific utilization indicators as the principal measure of program success, additional biases are introduced that may cloud evaluation results. This paper presents a non-technical introduction to time-series analysis (using disease-specific utilization measures) as an alternative, and more appropriate, approach to evaluating DM program effectiveness than the current total population approach.

Interrupted time series analysis in clinical research

OBJECTIVE

To demonstrate the usefulness of interrupted time series analysis in clinical trial design.

METHODS

A safety data set of electrocardiographic (ECG) information was simulated from actual data that had been collected in a Phase I study. Simulated data on 18 healthy volunteers based on a study performed in a contract research facility were collected based on single doses of an experimental medication that may affect ECG parameters. Serial ECGs were collected before and during treatment with the experimental medication. Data from 7 real subjects receiving placebo were used to simulate the pretreatment phase of time series; data from 18 real subjects receiving active treatment were used to simulate the treatment phase of the time series. Visual inspection of data was performed, followed by tests for trend, seasonality, and autocorrelation by use of SAS.

RESULTS

There was no evidence of trend, seasonality, or autocorrelation. In 11 of 18 simulated individuals, statistically significant changes in QTc intervals were observed following treatment with the experimental medication. A significant time of day and treatment interaction was observed in 4 simulated patients.

CONCLUSIONS

Interrupted time series analysis techniques offer an additional tool for the study of clinical situations in which patients must act as their own controls and where serial data can be collected at evenly distributed intervals.