Interact CardioVasc Thorac Surg 2006;5:15-17. doi:10.1510/icvts.2005.122705
© 2006 European Association of Cardio-Thoracic Surgery
Brief communication - Cardiac general |
Does the logistic EuroSCORE offer an advantage over the additive model?
Ruyun Jin* and
Gary L. Grunkemeier
For the Providence Health System Cardiovascular Study Group, Providence Health System, Portland, Oregon, USA
Received 10 October 2005;
accepted 17 October 2005
*Corresponding author: Ruyun Jin, MD, Providence St. Vincent Hospital and Medical Center, 9205 SW Barnes Road, LL#33, Portland, Oregon 97225, USA. Tel.: +1-503-216-7276; fax: +1-503-216-7274.
E-mail address: Ruyun.Jin{at}providence.org (R. Jin).
 |
Abstract
|
|---|
There are two versions of the EuroSCORE cardiac surgery risk model for mortality: the logistic regression which provides a predicted probability of death, and an additive score, a simplified approximation to the logistic probability, which can be easily calculated without a computing aid. A recent comparison of these two models' performance concluded that the logistic probability did not offer a distinct advantage over the additive score, and that the additive score was just as accurate as the logistic probability, even in high-risk patients. This conflicts with the conclusion of our previous study of the same issue. Thus, we did further analyses of our original data, following the approach of these authors. The additive and logistic EuroSCORE models were retrospectively applied to predict operative mortality in 23,463 cardiac surgery patients operated from 1997–2004 at Providence Health System hospitals. Both the logistic and additive risk predictions are higher than the observed mortality, and were calibrated downward to reconcile this difference. The observed-to-predicted ratio is relatively constant for the logistic model, but it varies with the risk score for the additive model. Thus, the logistic model can be recalibrated to fit the actual outcome well, but the recalibrated additive model cannot. The (recalibrated) logistic model is more accurate for different patient groups, including the high-risk patients, and should be preferred for clinical use.
Key Words: Logistic model; Additive model; EuroSCORE; Cardiac surgery; Mortality
|
1. Introduction
|
|---|
The EuroSCORE cardiac surgery risk model for mortality comes in two versions: a probability of death derived from a logistic regression model [1], and an additive score, a simplified approximation to the logistic probability [2], which can be calculated by hand, without any computing aid. A recent comparison of these two models performance by Shanmugam et al. concluded that ‘the logistic EuroSCORE did not offer a distinct advantage over the additive model and the logistic EuroSCORE was not more accurate even in high-risk patients’ [3]. Our recent study of this same topic came to the opposite conclusion: that the additive score is inferior to the logistic probability, and that it underestimates the probability of death for high-risk patients [4].
Our original approach was to consider the logistic result as the true probability and to show that the additive score was a poor approximation of it. Shanmugam et al. used another approach, to consider the observed mortality as the truth and to compare both EuroSCORE models to it. We used the ideas generated by this study to perform further analyses to support our previous conclusions, that the logistic probability is superior to the additive score and should be used whenever possible. Three critical aspects of their analyses were re-visited: (1) The logistic EuroSCORE over-predicted the mortality in their patient population (5.2% expected, 2.95% observed), yet it was not recalibrated to adjust for this baseline difference; (2) The discrepancy between the observed (O) mortality and the expected (E) mortality predicted by the model was measured by their difference (O-E), rather than by their ratio (O/E), so that high-risk patients would tend to have larger differences; (3) the patient groupings used for the analyses were defined by the additive scores rather than the logistic probabilities.
|
2. Materials and methods
|
|---|
The additive and logistic EuroSCORE models were applied to predict operative mortality for 23,463 patients who underwent cardiac surgery from 1997 to 2004 in the Providence Health System hospitals [4]. There were 69% CABG surgeries, 15% valve, 13% CABG plus valve and 3% other cardiac procedures, with a total of 852 operative deaths, or 3.6% mortality. Both EuroSCORE methods over-predicted the mortality: the logistic probability predicted 1944.5 deaths (the sum of the individual predictions), or 8.3% mortality, and the additive score predicted 1261.7 deaths, or 5.4% mortality.
To make the overall predictions aligning with the observed mortality, we calibrated the EurosSCORE predictions. The recommended way is by performing a new logistic regression with the logit of the probability as the only risk factor [5]. Since this approach would not work with the additive score, a simpler calculation was used to calibrate both the logistic and additive probabilities: the calibrated probability was set equal to the predicted probability times the ratio of total observed deaths to predicted deaths. Thus we multiplied each logistic probability by 3.6/8.3=0.438 and each additive score by 3.6/5.4=0.675. Statistical analyses were performed with S-PLUS version 6.1 (Insightful Corporation, Seattle, WA).
|
3. Results
|
|---|
The patients were grouped by their logistic probabilities into 10 equal sized subgroups. Fig. 1 (upper panel) shows the observed and the (uncalibrated) EuroSCORE mortality by subgroup. The logistic probabilitiy is higher than the observed mortality in all subgroups. The additive score is higher than the observed mortality except in the highest risk subgroup. The O/E ratio for the logistic model is relatively stable for all subgroups, but the O/E ratio for the additive model is not (Fig. 1, lower panel). The stability of the O/E ratio for the logistic probability implies that a recalibration will produce a good fit to the observed mortality.
View larger version (23K):
[in this window]
[in a new window]
|
Fig. 1. The observed and expected EuroSCORE mortalities by 10 subgroups containing equal number of patients sorted by the logistic probability (upper panel) and the ratios of observed to expected mortality (O/E) (lower panel). The x-axis shows the cut-points for each of the subgroups.
|
|
Fig. 2 shows the same analyses using the recalibrated EuroSCORE. The upper panel shows that the recalibrated logistic mortality is closer to the observed than the additive score. The lower panel compares the O/E ratios for both EuroSCORE models; the O/E ratio of the recalibrated logistic probability is closer to 1 than the additive score.
View larger version (23K):
[in this window]
[in a new window]
|
Fig. 2. The observed and recalibrated expected EuroSCORE mortalities by 10 subgroups containing equal number of patients sorted by the logistic probability (upper panel) and the ratios of observed to recalibrated expected mortality (O/E) (lower panel). The x-axis shows the cut-points for each of the subgroups.
|
|
|
4. Discussion
|
|---|
Shanmugam and colleagues compared the additive and logistic EuroSCOREs by presenting the O-E mortality for different risk groups [3]. They concluded that the additive EuroSCORE is a simple and easily applied system of risk assessment and that the logistic EuroSCORE does not offer a distinct advantage over the additive model, even in high-risk patients.
We respectfully disagree. In the present study, the O/E ratios are approximately constant for the logistic model but not so for the additive model (Fig. 1). Although the discrimination of the EuroSCORE, as measured by the ROC statistic, is excellent, as shown in our data and that of many others, it appears to consistently overestimate the mortality risk [3,6]. When the discrimination is good but the calibration is not, the model can be made more accurate by recalibration (although the opposite is not true) [7]. After recalibration, the logistic model fits the actual outcome better than the additive model (Fig. 2). This is a supernumerary analysis to confirm that the logistic model performs better than the additive model, and that the additive model underestimates the risk for high-risk patients, as shown previously using other statistical approaches [4]. Also, it appears from the data and figures presented in Shanmugam's paper [3], that using this same analysis on their data would lead to the same conclusion.
|
5. Conclusions
|
|---|
The logistic model has a relatively constant O/E ratio across risk groups, while the additive model does not. Thus, the recalibrated logistic model fits the actual outcome well, but the recalibrated additive model does not. The logistic model is more accurate for different patient groups and should be the primary choice for clinical use and group comparisons.
|
Acknowledgements
|
|---|
The following hospitals provided patient data: Providence Anchorage Medical Center, Anchorage, Alaska; Providence Everett Medical Center, Everett, Washington; Providence St. Peter Hospital, Olympia, Washington; Providence Yakima Medical Center, Yakima, Washington; Providence Portland Medical Center, Portland, Oregon; Providence St. Vincent Medical Center, Portland, Oregon; Providence St. Joseph Medical Center, Burbank, California; Providence Holy Cross Medical Center, Mission Hills, California; Providence Little Company of Mary Hospital, Torrance, California.
|
References
|
|---|
- Roques F, Michel P, Goldstone AR, Nashef SA. The logistic EuroSCORE. Eur Heart J 2003; 24:881–882.[Free Full Text]
- Nashef SA, Roques F, Michel P, Gauducheau E, Lemeshow S, Salamon R. European system for cardiac operative risk evaluation (EuroSCORE). Eur J Cardiothorac Surg 1999; 16:9–13.[Abstract/Free Full Text]
- Shanmugam G, West M, Berg G. Additive and logistic EuroSCORE performance in high risk patients. Interact CardioVasc Thorac Surg 2005; 4:299–303.[Abstract/Free Full Text]
- Jin R, Grunkemeier GL. Additive vs. logistic risk models for cardiac surgery mortality. Eur J Cardiothorac Surg 2005; 28:240–243.[Abstract/Free Full Text]
- Peterson ED, DeLong ER, Muhlbaier LH, Rosen AB, Buell HE, Kiefe CI, Kresowik TF. Challenges in comparing risk-adjusted bypass surgery mortality results: results from the Cooperative Cardiovascular Project. J Am Coll Cardiol 2000; 36:2174–2184.[Abstract/Free Full Text]
- Yap CH, Mohajeri M, Ihle BU, Wilson AC, Goyal S, Yii M. Validation of EuroSCORE model in an Australian patient population. ANZ J Surg 2005; 75:508–512.[CrossRef][Medline]
- Harrell FE Jr, Lee KL, Califf RM, Pryor DB, Rosati RA. Regression modelling strategies for improved prognostic prediction. Stat Med 1984; 3:143–152.[Medline]
This article has been cited by other articles:
|
|
|
|
 
M. Ranucci, S. Castelvecchio, L. Menicanti, A. Frigiola, and G. Pelissero
Risk of Assessing Mortality Risk in Elective Cardiac Operations: Age, Creatinine, Ejection Fraction, and the Law of Parsimony
Circulation,
June 23, 2009;
119(24):
3053 - 3061.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
A. Parolari, L. L. Pesce, M. Trezzi, C. Loardi, S. Kassem, C. Brambillasca, B. Miguel, E. Tremoli, P. Biglioli, and F. Alamanni
Performance of EuroSCORE in CABG and off-pump coronary artery bypass grafting: single institution experience and meta-analysis
Eur. Heart J.,
February 1, 2009;
30(3):
297 - 304.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
C. A. Mestres, M. A. Castro, E. Bernabeu, M. Josa, R. Cartana, J. L. Pomar, J. M. Miro, J. Mulet, and the Hospital Clinico Endocarditis Study Group
Preoperative risk stratification in infective endocarditis. Does the EuroSCORE model work? Preliminary results
Eur. J. Cardiothorac. Surg.,
August 1, 2007;
32(2):
281 - 285.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
Top
Abstract
1. Introduction