Application of Variable Life Adjusted Display (VLAD) to Risk-Adjusted Mortality of Esophagogastric Cancer Surgery

Integrating continuous monitoring and evaluation of risk-adjusted outcomes in a cardiac surgical program

BACKGROUND

The variable life-adjusted display (VLAD) method shows the difference between predicted and observed outcomes over time. Our study aims to implement routine in-house monitoring of risk-adjusted 30-day mortality and morbidity following cardiac surgery.

METHODS

The Society of Thoracic Surgeons (STS) risk score was calculated for 249 isolated and combined coronary and aortic or mitral valve cases performed during a 6-month period. The nine predicted STS variables were operative mortality, permanent stroke, renal failure (RF), prolonged ventilation, deep sternal wound (DSW) infection, reoperation for any reason, short and long length of stay (LOS), and major morbidity or operative mortality. EuroSCORE II was also calculated for the study population. VLAD plots were generated for each variable indicating whether performance is better or worse than expected on the basis of predicted risk of failure.

RESULTS

The mortality plot was fluctuating close to baseline risk. The prolonged ventilation, RF, reoperation, morbidity/mortality, and LOS plots were consistently positive, indicating favorable results. The stroke chart showed an upward trend for most of the period until two incidents toward last month led to a steep descent. The DSW infections plot though, indicated a worse-than-expected performance. The VLAD charts were shared in multidisciplinary meetings and clinicians were able to confront the performance with the population-specific expectancies and respond to adverse trends with further actions.

CONCLUSION

Graphical tool monitoring of risk-adjusted 30-day mortality and morbidity following cardiac surgery is feasible and allows detection of underperformance and implementation of changes in clinical practice.

A simple signaling rule for variable life-adjusted display derived from an equivalent risk-adjusted CUSUM chart

The variable life-adjusted display (VLAD) is the first risk-adjusted graphical procedure proposed in the literature for monitoring the performance of a surgeon. It displays the cumulative sum of expected minus observed deaths. It has since become highly popular because the statistic plotted is easy to understand. But it is also easy to misinterpret a surgeon's performance by utilizing the VLAD, potentially leading to grave consequences. The problem of misinterpretation is essentially caused by the variance of the VLAD's statistic that increases with sample size. In order for the VLAD to be truly useful, a simple signaling rule is desperately needed. Various forms of signaling rules have been developed, but they are usually quite complicated. Without signaling rules, making inferences using the VLAD alone is difficult if not misleading. In this paper, we establish an equivalence between a VLAD with V-mask and a risk-adjusted cumulative sum (RA-CUSUM) chart based on the difference between the estimated probability of death and surgical outcome. Average run length analysis based on simulation shows that this particular RA-CUSUM chart has similar performance as compared to the established RA-CUSUM chart based on the log-likelihood ratio statistic obtained by testing the odds ratio of death. We provide a simple design procedure for determining the V-mask parameters based on a resampling approach. Resampling from a real data set ensures that these parameters can be estimated appropriately. Finally, we illustrate the monitoring of a real surgeon's performance using VLAD with V-mask.

Monitoring the quality of cardiac surgery based on three or more surgical outcomes using a new variable life-adjusted display

Quality problem or issue

The traditional variable life-adjusted display (VLAD) is a graphical display of the difference between expected and actual cumulative deaths. The VLAD assumes binary outcomes: death within 30 days of an operation or survival beyond 30 days. Full recovery and bedridden for life, for example, are considered the same outcome. This binary classification results in a great loss of information.

Initial assessment

Although there are many grades of survival, the binary outcomes are commonly used to classify surgical outcomes. Consequently, quality monitoring procedures are developed based on binary outcomes. With a more refined set of outcomes, the sensitivities of these procedures can be expected to improve.

Choice of solution

A likelihood ratio method is used to define a penalty-reward scoring system based on three or more surgical outcomes for the new VLAD. The likelihood ratio statistic W is based on testing the odds ratio of cumulative probabilities of recovery R. Two methods of implementing the new VLAD are proposed.

Implementation

We accumulate the statistic W-W¯R to estimate the performance of a surgeon where W¯R is the average of the W's of a historical data set. The accumulated sum will be zero based on the historical data set. This ensures that if a new VLAD is plotted for a future surgeon of performance similar to this average performance, the plot will exhibit a horizontal trend.

Evaluation

For illustration of the new VLAD, we consider 3-outcome surgical results: death within 30 days, partial and full recoveries. In our first illustration, we show the effect of partial recoveries on surgical results of a surgeon. In our second and third illustrations, the surgical results of two surgeons are compared using both the traditional VLAD based on binary-outcome data and the new VLAD based on 3-outcome data. A reversal in relative performance of surgeons is observed when the new VLAD is used. In our final illustration, we display the surgical results of four surgeons using the new VLAD based completely on 3-outcome data.

Lessons learned

Full recovery and bedridden for life are two completely different outcomes. There is a great loss of information when different grades of 'successful' operations are naively classified as survival. When surgical outcomes are classified more accurately into more than two categories, the resulting new VLAD will reveal more accurately and fairly the surgical results.

Debate: should we use variable adjusted life displays (VLAD) to identify variations in performance in general surgery?

BACKGROUND

The recent push for the publication of individual surgeon outcomes underpins public interest in safer surgery. Conventional, retrospective assessment of surgical performance without continuous monitoring may lead to delays in identifying poor performance or recognition of practices that lead to be better than expected performance.

DISCUSSION

The variable life adjusted display (VLAD) is not new, yet is not widely utilised in General Surgery. Its construction is simple and if caveats are appreciated the interpretation is straightforward, allowing for continuous surveillance of surgical performance.

SUMMARY

While limitations in the detection of variations in performance are appreciated, the VLAD could represent a more useful tool for monitoring performance.