| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
|
© 2003 European Association of Cardio-Thoracic Surgery
Derivation and validation of a clinical scoring system to predict the need for an intra-aortic balloon pump in patients undergoing adult cardiac surgery
a Emergency Medicine Research Group, Department of Emergency Medicine, Manchester Royal Infirmary, Oxford Road, Manchester M13 9WL, UK
* Corresponding author. Tel.: +44-780-154-8122; fax: +44-161-276-8538 Received June 16, 2003; received in revised form August 12, 2003; accepted August 20, 2003
The spectrum of patients receiving cardiac surgery are increasing in age and severity of illness. With the reduction of complications caused by the placement of an intra-aortic balloon pump (IABP) there is increasing interest in the placement of an IABP prophylactically. We sought to derive a scoring system to guide the placement of IABPs. A total of 3927 patients from the Blackpool Victoria Open Heart Registry were used to derive a range of clinical decision scores using a range of established and novel statistical techniques. This database included 127 patients who received an IABP. The derived scores and rules were then validated on the North Staffordshire Open Heart Registry, containing 3070 patients, and 161 patients who received an IABP. We derived and validated a clinical score that has a sensitivity of 50% and a specificity of 96.5% in the prediction of those patients requiring an IABP. This was robust in the validation dataset and outperformed the Parsonnet score in this context. Our validated clinical scoring system will be useful both to guide individual clinical decision making and to compare variation of IABP usage among institutions.
Key Words: Intra-aortic balloon pumping; Clinical protocol; Thoracic surgery
Several major scoring systems currently exist that perform well in stratifying patients for risk of suffering death after cardiac surgery [1–3]. These scoring systems have been invaluable both in the decision-making process for individual patients and also for quality assessment and comparisons of novel and established techniques in cardiac surgery. However, while these risk scores perform well in low and moderate risk patients, these scores often perform less well in high-risk patients [3]. Furthermore, these scores are not designed to guide specific treatment decisions in the high-risk patient and although studies have shown reduced hospital stay, shorter intensive cre unit (ICU) stay and increased survival in high-risk patients who receive a pre-operative balloon pump [4,5–9], no scoring system exists to guide or compare treatment in patients being considered for an intra-aortic balloon pump (IABP). Many clinicians believe that a prophylactically inserted IABP may be of great benefit in lower risk patients, but cohort studies in this area suffer greatly from the fact that those receiving an IABP and those not receiving an IABP differ markedly in their clinical status. Thus even with logistic regression which can go some way to accounting for differences between groups, it is still very difficult to come to firm conclusions in such studies. Far more preferable would be a well-conducted randomized controlled study. However, there is currently no well-defined way to select the group of patients for entry into such a study in whom an IABP may most benefit. Therefore in order to further investigate these patients either by risk matching, or more preferably by a well-conducted randomized controlled trial, a clinical risk score will be vital in order to correctly identify the groups most likely to benefit from an IABP. To derive a clinical risk score that may assist in the identification of patients that would benefit from a balloon pump pre-operatively, we sought to use a range of current and novel statistical techniques to obtain an optimal clinical score. We feel that a scoring system will be an invaluable tool to guide pre-operative IABP placement for prospective studies into early IABP placement, and will also be useful to compare differences in treatment and outcomes in high-risk patients among institutions.
2.1. Study setting The Blackpool Open Heart Registry was used from 1996 to 2000 to derive rules. This is a UK centre performing 900 operations per year in the north-west of England. The rules were then validated on the Stoke cardiac database from 1997–2002, a second centre in the UK that performs a similar number of procedures. 2.2. Outcomes IABPs inserted either preoperatively or peri-operatively were classified as positive cases for the purposes of our study.2.3. Statistical analysis Univariate analysis was used initially to identify significant variables that predict the placement of an IABP in this population. The Pearson's chi-squared test was used for categorical variables and the unpaired Student t-test was used for continuous variables if a normal distribution for that variable could be shown. Non-parametric continuous variables were analysed using the Mann–Whitney U-test. Variables that were significant at the level and that had less than 20% missing data-points were included in multivariate analysis. Logistic regression was then used to derive models to predict the placement of an IABP. Standard logistic regression methods were initially used and then conditional forward regression and backward regression was used in an attempt to improve the model. Pearson's correlation coefficients were used to assess all variables for correlation. Models were assessed for goodness-of-fit using receiving operator characteristic (ROC) curves and the Hosmer–Lemeshow test. Simple additive models were derived from the odds ratios, and a further complex logistic model was derived with the optimal rule to see if this increased complexity provided a superior model. This methodology is similar to that used in the derivation of the Euroscore [2]. A Baysian model [3] and a Recursive Partitioning model [10,11] were also calculated to see if any further improvement in the score could be achieved. Univariate analysis and multivariate analysis was performed using SPSS version 10.1 and STATA version 7. Recursive partitioning was performed using CART version 4. All decision rules were validated on the Stoke database using ROC curves, sensitivity and specificity. Optimal rules were chosen according to their ability to predict the placement of an IABP. Cut-off points were chosen at the point at which 50% of those patients who received IABPs were correctly identified. This cut-off point was decided clinically using the standpoint that this would be the minimally acceptable sensitivity for such a rule to be clinically useful.
3.1. Sample data The results of 3921 patients were analysed from the Blackpool Victoria Hospital Open Heart Registry from January 1996 to December 1999. This was used as the derivation sample. A total of 3070 patients was analysed from the North Staffordshire Hospital Open Heart Registry, from March 1998 to May 2002. This was used to assess the external validity of the derived rules. A descriptive analysis of both these databases is given in Table 1.
3.2. Assessment of existing rules The Parsonnet score was first assessed for its ability to predict the need for an IABP in both datasets. The Parsonnet score gave an area under the ROC curve of 0.715 and 0.687 for the Blackpool and North Staffordshire databases, respectively. However, when the cut-off point that identified 50% of those patients that had an IABP was identified (at a Parsonnet score of 10), the Parsonnet misclassified 30 and 29%, respectively, of all patients who did not require an IABP in these databases, therefore limiting severely the ability of using a Parsonnet score of over 10 to influence the decision to insert an IABP. Neither database contained adequate data for assessment of the Euroscore. 3.3. Univariate analysis The Blackpool Victoria Hospital database was analysed to find univariate predictors for the placement of an IABP. Variables that showed a statistically significant difference between the IABP and non-IABP groups with a P-value of 0.2 or lower were selected as candidate variables to enter in a model, if the level of missing variables in that category was less than 20%. Twenty-seven candidate variables were found by this method.3.4. A logistic regression model All selected variables were used to create a logistic regression model with IABP as the dichotomous dependent variable. This gives a multivariate coefficient for each variable with a standard error. All variables with a non-significant coefficient at the 0.05 level were excluded. The remaining coefficients were converted into relative odds ratio (by calculating eB). In order to create a simple summative predictive score these ratios were used as the score assigned to each variable. The clinical scoring system is given in Table 2 and its performance is demonstrated in Table 3.
In order to verify that our selected variables were not over-correlated, which would result in our rule containing variables that were not adding any further useful information to the scores for each patient, we used the Pearson correlation coefficient. No variable correlated with another above a level of 0.583, which means that each variable is selecting significantly different patients for higher scores. We also conclude that important predictive information would be lost if any variables were left out. This Backward Logistic Regression model outperformed all other models including any Complex logistic models, Baysian models or Recursive Partitioning models in all methods of assessment (and therefore no further description of these inferior rules are given). The Hosmer–Lemeshow test gave a P-value of 0.929, and on assessment of its predictive ability as applied to both databases the area under the ROC curve was 0.823 and 0.846, respectively. A cut-off point of 10 predicted 50% of those that had an IABP in the Blackpool database, and misclassified only 3.5 and 2.6% of patients not requiring an IABP in the Blackpool and North Staffordshire databases. This rule therefore misclassified the lowest number of patients in both databases and with a total patient number of 6991 in both databases, this rule selected 153 of the 288 patients that had an IABP and only selected 211 as high risk out of the 6703 patients who did not have an IABP. An analysis of standardized residuals after model prediction showed that it was unlikely that we had omitted any variables that would have added to the predictive ability of this scoring system.
We have found a decision score that uses ten common and easily available variables to give a score that performs well in giving an assessment of the probability of needing an IABP in patients prior to cardiac surgery. A score of above 10 predicts 50% of patients that went on to require a balloon pump, with a specificity of 96.5%. We have shown that it outperforms current scoring systems and that no extra predictive power is obtained if a complex logistic model is used and the difficult to calculate but mathematically correct formula is used in place of our simple rule. It must be realized that in contrast to the outcomes of death or stroke, the insertion of an IABP is a variable that is controlled by the decisions made by clinicians. In particular the absence of guidelines in this area will mean that the decision to insert an IABP may vary greatly among clinicians. This is a weakness of this scoring system. However, by including patients who received an IABP intra-operatively our scoring system will also identify those patients not identified by clinicians preoperatively. Looking for an alternative outcome measure to IABP insertion is difficult. We believe that there is no single, objective measure that could take its place for the identification of high-risk patients pre-operatively who require an IABP. Cardiac Index, which would perhaps be a more objective measure, is not available in the majority of patients in this context. In addition the Cardiac Index varies greatly during Cardiac Surgery and no single point of measurement could be used to decide if an IABP would be necessary. Any other outcomes such as in-hospital mortality would not be valid in predicting the need for a pre-operative IABP, as not all deaths would have benefited from an IABP. Our scoring system may therefore be regarded as a propensity score in this context [12]. Our intention is to reflect clinical practice in this score so that guidelines can be constructed and institutions compared, and therefore we do not feel that this is a weakness. Our score uses ten variables including inotrope usage, cardiogenic shock, priority, left main stem disease, ejection fraction, re-do operation, and recent catheterization to predict the need for an IABP. Holman et al. [9] derived a propensity score to use for patient matching in their study to investigate the use of prophylactic IABP use. They used re-do operation and left main stem disease as in our model; however, all other variables used in our model were used as an exclusion criteria in their study as they were investigating haemodynamically stable patients. Their study did not find a mortality benefit with prophylactic IABP use, and our study demonstrates that on the whole they were investigating patients who would not usually receive an IABP in our centres.
Our results correlate with studies by Christenson et al. and Dietl et al. [4–7]. Christenson's randomized controlled trial of 60 patients with any two criteria of: re-do operation, ejection fraction Our scoring system is the first validated, comprehensive scoring system for the prediction of patients that may benefit form IABP. This will be of great use to clinicians seeking to investigate the use of the IABP in cohort studies and will be vital to future studies seeking to identify high-risk patients for inclusion into randomized controlled trials for pre-operative IABP insertion. In addition, this scoring system is the first risk score that will allow clinicians to assess their own practice of IABP insertion with others both nationally and internationally.
The rule we have derived is robust in a second hospital and provides important data on our highest risk patients that can guide decision-making and also audit practice in these patients. We hope to further validate our score in the full UK Dataset and to use the scoring system in a future randomized controlled trial for preoperative IABP insertion.
ICVTS on-line discussion Author: Dr. Sameh Sersar, Assistant lecturer, Cardiothoracic surgery, Mansoura University, Mansoura 123, Egypt Date: 13-Sep-2003 Message: Let me ask about the value of the optimal system. In other words, at which score you must or you prefer to use the prophylactic intra aortic balloon. Christensen et al. (1999) indicated at least two conditions of the following: ejection fraction less than 30% (you gave it 8), left main disease (you gave a 2 score), unstable angina or reoperation (you gave 2). So is the score used for diagnostic or prognostic values? What do you mean by salvage priority? Response Author: Mr. Joel Dunning, Royal College of Surgeons Research Fellow, Dept Cardiothoracic Surgery, Manchester Royal Infirmary, Oxford Road, Manchester M13 3BW, UK. Date: 30-Sep-2003 Message: In answer to the question about salvage priority, this is defined as patients requiring CPR en-route to the operating theatre or prior to anaesthetic induction (CPR following anaesthetic induction should not be included.) This is the definition given by the Society of Cardiothoracic Surgeons (SCTS) for the UK national dataset. In answer to the question as to how our scoring system should be used: Unlike Christenson who gives one rule and presents it for use as a dichotomous decision rule : i.e. insert an IABP or do not insert an IABP, our rule is intended for more flexible usage. Surgeons can retrospectively analyse their own databases and decide for themselves their own cut off point at which they would decide to use an IABP. In addition, the current literature that attempts to define the cohort of patients who would benefit from prophylactic IABP insertion is poor. We would like to embark on a prospective study in this area. However, the most difficult part of this study is to accurately define the patients that are most likely to benefit from IABP insertion. This scoring system will allow us and other researchers to decide on a cut-off point in this scoring system and to allocate these groups of patients into prospective research. It is hoped that this would then allow studies to clearly demonstrate a survival advantage for particular subgroups of patients undergoing Cardiac Surgery.
J.D. is funded by the Enid Linder Research Foundation (a non-medical charity) Fellowship from the Royal College of Surgeons of England. We are also grateful for the assistance of Maninder Kalkat in the writing of this paper. No commercial funding has been provided for this study. doi:10.1016/S1010-7940(03)00569-4
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
| ANN THORAC SURG | ASIAN CARDIOVASC THORAC ANN | EUR J CARDIOTHORAC SURG |
| J THORAC CARDIOVASC SURG | ICVTS | ALL CTSNet JOURNALS |
Top
Abstract
1. Introduction
, left main stem stenosis 
, or unstable angina, demonstrated lower bypass times, shorter ICU and hospital stays, and a lower hospital mortality if an IABP was used pre-operatively. Dietl's retrospective cohort study of 163 patients with an ejection fraction of less than 25%, identified re-do operation, left main stem disease, LVEDP 
mmHg, non-elective operation and NYHA class III or IV as predictors of 30-day mortality for patients undergoing revascularisation with an ejection fraction 
. They also demonstrated a mortality benefit for pre-operative IABP use in these patients. Our clinical score would assign high scores to patients who have been identified in these studies.