Interact CardioVasc Thorac Surg 2007;6:470-473. doi:10.1510/icvts.2007.153056
© 2007 European Association of Cardio-Thoracic Surgery
Institutional report - Cardiac general |
Evaluation of cardiac scoring models for an Austrian cardiac register
Oskar Staudingera,*,
Herwig Ostermanna,
Günther Lauferc,
Roland Schistekb,
Bettina Staudingera and
Bernhard Tilga
a UMIT, University for Health Sciences, Medical Informatics and Technology, Hall i.T., Austria
b Landeskliniken Salzburg Herzchirurgie
c Univ.-Klinik für Chirurgie (MUI), Klin. Abteilung für Herzchirurgie, Innsbruck, Austria
Received 24 January 2007;
received in revised form 3 April 2007;
accepted 3 April 2007
 The Cardiac Project was carried out with the financial support of the Austrian Research Promotion Agency (FFG) and the Tyrolean Future Foundation, supported under the competence centre hitt – health information technologies tirol.
*Corresponding author. c/o UMIT, A. Wallnöfer-Zentrum 1, A-6060 Hall i.T., Austria. Tel.: +43 660 2208 300; fax: +43 7252 2208 14.
E-mail address: os{at}s2-engineering.com (O. Staudinger).
 |
Abstract
|
|---|
The Austrian Health 2006 Structural Plan of 28 June 2006 requires that reference centres for heart surgery ‘participate in result-quality registers (e.g. cardiac registers)’. The aim of the present study was to identify a suitable scoring model for the Austrian register during the run-up to its creation. During the period from November 2004 to December 2005 a survey was done of the actual situation, the organisational and economic possibilities, and the requirements of all Austrian heart centres. General and also specific Austrian basic conditions were defined. Scoring models were then classified and evaluated. The characteristics ‘national and international comparability’, the associated ‘distribution of the scoring system’ and ‘detailed scientific discussion of the applicability’ were found as the main criteria for selection. Economic aspects such as survey and analysis costs, and the everyday practicability of gathering data in the actual situation revealed by the survey, were also included in the evaluation framework. It could be demonstrated that under the given circumstances, the EuroSCORE represented the suitable predictive model.
Key Words: Cardiac database; Risk stratification; Scoring model
|
1. Introduction
|
|---|
National heart registers were set up in many countries, for example the USA, Canada, UK, Australia, etc. In the USA and UK, publication of the outcome is already implemented down to the level of individual surgeons. The Austrian Heart Register was established during the period October 2004 to February 2006 within a project sponsored by the FFG (Austrian Research Promotion Agency). Within the framework of this research project the primary objective of the register was defined as follows: ‘Collecting data and making this accessible for national and international benchmarking of cardiac surgical interventions in patients over 18 years of age in Austria, enabling the data to be analysed and in this way establishing a basis for improving result quality in the heart centres’.
|
2. Materials and methods
|
|---|
A three-phase procedure was chosen:- The nine Austrian heart centres were subjected to a closed, structured status survey, which contained six items based on established concepts.
- In addition to medical requirements, technical and economic criteria were defined since the selection was intended to lead to the basis for an extensive index that could be used in practice.
- Models revealed by a search of the literature were added to the mutually agreed predictive models proposed in the second phase. These were evaluated with reference to the previously defined criteria and the specification of the initial scoring model for the Austrian Heart Register was defined.
|
3. Status survey in the Austrian heart centres
|
|---|
To determine the basic parameters for defining the objectives and, therefore, the selection of the suitable predictive model, the current survey methods and the scope of the data gathered during the period October 2004 to October 2005 was surveyed in all nine heart centres using a closed, structured questionnaire. All nine Austrian heart centres took part in this survey (three university hospitals and six other hospitals). The following items were surveyed (Table 1).
|
4. Interpretation of the results with reference to criteria for defining the objective
|
|---|
The study revealed that the survey of risk factors for calculating the risk adjusted expected mortality in the Austrian heart centres was inhomogeneous both in terms of the way it was organised and in terms of the scope and nature of the data gathered. It may be stated that, overall, the recording and calculation of the risk adjusted expected mortality was performed as a non-standardised process.
|
5. Selection criteria for the predictive model
|
|---|
The essentials of the main basic parameters for the general specifications of a scoring model were already defined in the FFG (Austrian Research Promotion Agency) application and relate primarily to the characteristics of and compatibility with the requirements of a public health system. This raises, on the one hand, the issue of transparency and the comparability associated with it and, on the other, the question of usage in practice in connection with the overall objective, which is also described in the literature on the introduction of national heart registers. The EuroSCORE and Parsonnet were defined as a minimum basis for the study.- The model must enable the result quality to be compared at the national and international level. Extensive discussion of the model in scientific publications was stipulated as a criterion.
- The model must be relevant for the Austrian population. This was then accepted by us if the model was already used in a large number of clinical case studies based on the predictive model.
- The model must be relevant for selected subgroups. Groups by major procedure type, by gender, by age and combinations of these must deliver significant predictive probabilities.
- The population of the model must agree with the target population in respect of age selection (adults from 18 years of age) and in respect of the reason for the intervention (acquired). Paediatric scores and scores for inherited heart conditions (congenital scores) are excluded.
- The scoring model must calculate a risk adjusted expected mortality as its result.
- The individual predictive significance is regarded as of secondary importance in choosing the model. Over- or underestimates of individual cases, e.g. in the case of unusual combinations of risk factors, are not yet defined per se as a reason for rejection [1].
- The number of relevant parameters to be gathered must be as small as possible, without impairing the quality of the prognosis.
- The basis for deciding whether a parameter is positive or negative, or the grade of a parameter, must be tightly defined. Too much leeway for evaluation would make recording difficult for non-medical personnel and would increase the costs of performing checks.
- Gathering the criteria must lead neither to an unacceptable increase in administration expenses as a result of additional personnel, nor to significant extra costs. This means that the criteria must be available for the most part from patient files, operation protocols and standard laboratory findings. An interesting study in this connection is that of Albert et al. in Germany from 2004 [2], in which costs of
 10 per data record are assumed. Although this topic has yet to be investigated in Austria, it may be assumed that costs (full costs) at the moment are not less than the above.
- The calculation method for the model must be publicly accessible and licence free.
|
6. Selection of the predictive model
|
|---|
In addition to the above scoring models, further scores were identified in a search of the literature and added to the list. In accordance with Criterion 4 and 5, all paediatric scores, congenital scores and scores for specific questions were excluded. The following scoring systems remained (Table 2).
No. 7 was excluded because the licence was not released; No. 6 was excluded because of the chronological variance of the model. No. 3 was not pursued further because of the large number of parameters. On the basis of the above classification, the EuroSCORE [3] (additive and logistic), Initial Parsonnet [4], French Score [5] and the Ontario Province Risk Score [6] were placed on the short list.
Studies were analysed which have demonstrated that, even for different populations, the EuroSCORE also comprises a suitable predictive instrument on the basis of the STS data set. Nashef confirmed that the EuroSCORE also achieves good predictive results on data from the STS database [7].
Roques and colleagues [8] have studied the applicability of the EuroSCORE in different European countries and came to the conclusion: ‘Despite epidemiological differences between European countries, the discriminative power of EuroSCORE was good in Spain and excellent in all other countries’.
Investigations of both the OPR [9] and also the French Score produced predominantly positive results [10]. However, in comparison with the EuroSCORE the number of publications and the description of use as the basis of clinical trials was significantly lower, which was evaluated as favouring the EuroSCORE.
In addition to a large number of positive reports on the use of the Parsonnet Score, e.g. in comparison with general scoring systems [11], publications were also found which questioned the predictive quality of the Parsonnet Score or even rejected it [12, 13]. A further argument against the Parsonnet Score is the possibility of partial interpretation of the risk factors, which does not exist in the other models; this score was, therefore, also excluded.
|
7. Discussion
|
|---|
Choosing the right period for observing the mortality and a suitable predictive model is of fundamental importance for benchmarking the outcome performance of the participating institutions:
The primary and, in reality, the most used criterion of the result of a cardiac surgical intervention is the 30-day mortality and nearly all scoring models, which provide national and international benchmarking to calculate this result.
The common criterion for benchmarking outcome quality is
| (1) |
Osswald and co-workers [14] recommend observation of a 3-month (for CABG), 6-month (for valves) or even a 1-year mortality (for combined procedures) instead of the 30-day mortality. As soon as a comparable, international proofed model (which calculates a 3-month, 6-month or 1-year predicted mortality) is available, the Austrian Cardiac Database will incorporate it parallel to the existing EuroSCORE. Concerning the figures of long-term mortality and long-term outcome quality, does this mean that gathering of the mortality date and simple statistical analyses without relation to the predicted mortality will be continued as it is already established. The short-term figures are based on the logistic EuroSCORE and reflect the outcome quality on it.
Since the objective of this study was the selection of a suitable scoring model for the Austrian Heart Register, the lack of available data at the time the work took place prevented any verification of the relevance and significance of the EuroSCORE using data from the whole of Austria. Since on the basis of the above criteria the EuroSCORE has been selected for the calculation of result quality, the demand must be made that this decision should also be secured from a medical aspect after sufficient data material has become available.
|
References
|
|---|
- Kurkia T, Järvinen O, Katajac MJ, Laurikkab J, Tarkkab M. Performance of three preoperative risk indices; CABDEAL, EuroSCORE and Cleveland models in a prospective coronary bypass database. Eur J Cardiothorac Surg 2002; 21:406–410.[Abstract/Free Full Text]
- Albert AA, Rosendahl UP, Ennker J, Freud J. 30-Tage-Follow-up in der Herzchirurgie: Methodik und Kosten. Gesundh ökon Qual manag 2004; 9:32–35.[CrossRef]
- Nashef SAM, Roques F, Michel P, Gauducheau E, Lemeshow S, Salmon R. European system for cardiac operative risk evaluation (EuroSCORE). Eur J Cardiothorac Surg 1999; 16:9–13.[Medline]
- Parsonnet V. A method of uniform stratification of risk evaluating the results of surgery in acquired adult heart disease. Circulation 1989; 79:I3–12.[Medline]
- Roques F, Gabrielle F, Michel P, De Vincentiis C, Baudet M. Quality of care in adult heart surgery: proposal for a self-assessment approach based on a French multicenter study. Eur J Cardiothorac Surg 1995; 9:433–440.[Abstract]
- Tu JV, Jaglal SB, Naylor D. Mulitcenter validation of a risk index for mortality, intensive care unit stay and overall hospital length of stay after cardiac surgery. Circulation 1995; 91:677–684.[Abstract/Free Full Text]
- Nashef SAM, Roques F, Hammill BG, Peterson ED, Michel PM, Grover FL, Wyse RK, Ferguson TB. Validation of European System for Cardiac Operative Risk Evaluation (EuroSCORE) in North American cardiac surgery. Eur J Cardiothorac Surg 2002; 22:101–105.[Abstract/Free Full Text]
- Roques F, Nashef SAM, Michel P, Pinna Pintor P, David M, Baudet E. The EuroSCORE Study Group. Does EuroSCORE work in individual European countries. Eur J Cardiothorac Surg 2000; 18:27–30.[Abstract/Free Full Text]
- Südkamp M, Geissler HJ, Hölzl P, de Vivie ER. Risikostratifizierung in der Herzchirurgie-Entscheidungshilfe bei der Indikationsstellung. Z Kardiol 2000; 89:667–673.[CrossRef][Medline]
- Habicht JM. Risikostratifizierung in der Herzchirurgie: welcher Score eignet sich am besten. ZaeFQ 2000; 94:745–749.
- Martínez-Alario J, Tuesta ID, Plasencia E, Santana M, Mora ML. Mortality prediction in cardiac surgery patients, comparative performance of Parsonnet and general severity systems. Circulation 1999; 99:2378–2382.[Abstract/Free Full Text]
- Gabrielle F, Roques F, Michel P, Bernard A, de Vicentis C, Roques X, Brenot R, Baudet E, David M. Is the Parsonnet's score a good predictive score of mortality in adult cardiac surgery: assessment by a French multicentre study. Eur J Cardiothorac Surg 1997; 11:406–414.[Abstract]
- Wynne-Jones K, Jackson M, Grotte G, Bridgewater B. on behalf of the North West Regional Cardiac Surgery Audit Steering Group. Limitations of the Parsonnet score for measuring risk stratified mortality in the north west of England. Heart 2000; 84:71–78.[Abstract/Free Full Text]
- Osswald BR, Blackstone EH, Tochtermann U, Thomas G, Vahl CF, Hagl S. The meaning of early mortality after CABG. Eur J CardioThorac Surg 1999; 15:401–407.[Abstract/Free Full Text]
Related Article
-
ICVTS on-line discussion A EuroSCORE and specialised databases
Interactive CardioVascular and Thoracic Surgery 6: 473-473.
[Full Text]
This article has been cited by other articles:
|
|
|
|
 
N. Hudorovic
ICVTS on-line discussion A EuroSCORE and specialised databases
Interactive CardioVascular and Thoracic Surgery,
August 1, 2007;
6(4):
473 - 473.
[Full Text]
[PDF]
|
|
|
|
|
Top
Abstract
1. Introduction
