This Article Abstract Full Text (PDF) On-line Video Alert me when this article is cited Alert me if a correction is posted Services Email this article to a friend Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Add to Personal Folders Download to citation manager Author home page(s): Sharif Al-Ruzzeh Shishir Karthik David O'Regan Permission Requests Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Al-Ruzzeh, S. Articles by O'Regan, D. Search for Related Content PubMed PubMed Citation Articles by Al-Ruzzeh, S. Articles by O'Regan, D. Related Collections Education Professional affairs

Objective surgical skill assessment: the diagonal operating matrix

Yorkshire Heart Centre, Leeds General Infirmary, Great George Street, Leeds LS1 3EX, UK

Received 28 July 2006; received in revised form 13 December 2006; accepted 14 December 2006

Presented at the joint 20th Annual Meeting of the European Association for Cardio-thoracic Surgery and the 14th Annual Meeting of the European Society of Thoracic Surgeons, Stockholm, Sweden, September 10–13, 2006.

*Corresponding author. Tel.: +44-113-3925790; fax: +44-113-3928092.

E-mail address: David.O'Regan{at}leedsth.nhs.uk (D. O'Regan).

	Abstract

Top Abstract 1. Introduction 2. Materials and methods 3. Discussion References

 
There is an urgent need for structured surgical training and assessment due to the reduction in the training duration with the European Working Time Directive (EWTD). We propose a model for objective skill assessment, the PAR-Diagonal Operating Matrix (PAR-DOM) which breaks down the task of vascular anastomosis into clearly defined skills. The PAR-DOM is made up of a 3x5 table and progress is made along vectors defined on the x-axis as PAR and on the y-axis as four levels. PAR defines three skills at each level. Each skill is graded from 1–3 (this may be taken as below average, average, above average). The skills at various levels are: Level 0 – Posture, Address, Relaxation; Level 1 – Pick-up, Airtime, Rotation; Level 2 – Placing, Angles, Rhythm; Level 3 – Precision, Adaptability, Reproducibility; Level 4 – Pace, Awareness, Relations. The PAR-DOM matrix provides a graphic representation of the progress of trainees over their training period assigned for them to stay with the trainer and also help identify individual strengths and weaknesses.

Key Words: Surgical skills; Training; Assessment

	1. Introduction

Top Abstract 1. Introduction 2. Materials and methods 3. Discussion References

 
Surgical training, in general, and cardiac surgical training, in particular, is passing through a critical or ‘turbulent’ time [1]. In the United Kingdom, the training bodies have become concerned about surgical training due to the reduced number of years of training with the adoption of the Calman system and reduced working hours with implementation of European Working Time Directive (EWTD) [2]. There is an urgent need to develop means of assessment in order to ensure the provision of an adequate and satisfactory training.

We propose the PAR-Diagonal Operating Matrix (PAR-DOM) as an objective assessment tool that breaks down the task of vascular anastomosis into clearly defined skills.

	2. Materials and methods

Top Abstract 1. Introduction 2. Materials and methods 3. Discussion References

 
2.1. The assessment model

View larger version (13K): [in this window] [in a new window]   Fig. 1. The PAR-DOM model.

1. Posture: Assesses the ability of the trainee to attain the optimal posture and height of the table, which will define the anatomical working angles of the shoulders, the elbows and hands. The shoulders should be relaxed with straight back, the elbows extended slightly beyond the 90° angle and the wrists relaxed in slight palmar flexion (Video 1).
   
2. Address: Assesses the ability to shift from right leg to left leg in order to alter the rotation of the hips, so the shoulders enable a forehand and backhand stitch, respectively, with the ability to move around the operating table (Video 2).
   
3. Relaxation: Assesses the ability to relax the body and move the arms, wrists and fingers independently as required as discrete separate movements. In other words, gross forearm or wrist movements do not substitute fine finger movements.
   

View larger version (124K): [in this window] [in a new window]   Video 1. Shows how the trainee can attain the optimal posture and height of the table, which will define the anatomical working angles of the shoulders, the elbows and hands. The shoulders should be relaxed with straight back, the elbows extended slightly beyond the 90° angle and the wrists relaxed in slight palmar flexion.

View larger version (93K): [in this window] [in a new window]   Video 2. Shows the ability to shift from right leg to left leg in order to alter the rotation of the hips, so the shoulders enable a forehand and backhand stitch, respectively, with the ability to move around the operating table.

1. Pick-up: Assesses the ability to pick up the needle accurately within the jaws of a needle holder and at the correct angle relative to the tissue to be stitched.
   
2. Airtime: Assesses the amount of time the trainee spends to adjust the angle of the needle on the needle holder to get ready for the next stitch. The trainee is advised not to use fingers to adjust the needle in position but rather use an instrument or surface as previously suggested [3].
   
3. Rotation: Assesses the ability to rotate the needle around a central point, such that the needle circumscribes a perfect circle throughout its path. This requires introduction of the needle at an angle of 90° on entering the tissue and on exiting from it. This aims to avoid dragging the heel of the needle through the tissue that can predispose to needle holes (Video 3).
   

View larger version (87K): [in this window] [in a new window]   Video 3. Shows the ability to rotate the needle around a central point, such that the needle circumscribes a perfect circle throughout its path.

1. Placing: Assesses the ability to place the needle in a correct position in relation to the tissue, as well as placing the suture accurately on the follow through (Video 4).
   
2. Angles: Assesses the ability to attain the suitable angle in relation to the needle holder so that the needle can be accurately rotated through the tissues (Video 4).
   
3. Rhythm: Assesses the ability to repeat the previous processes rhythmically. This becomes dependent on the smooth rotation of the needle on entry into the tissue and on exit to facilitate ease of pick up. The emphasis is on economy of time and movement.
   

View larger version (102K): [in this window] [in a new window]   Video 4. Shows the ability to place the needle in a correct position in relation to the tissue and the ability to attain the suitable angle in relation to the needle holder so that the needle can be accurately rotated through the tissues.

1. Precision: Assesses the ability to do the previous needle exercises of introducing the needle through the tissue and placing the suture accurately whilst using the working angles precisely.
   
2. Adaptability: Assesses the ability to actually stitch in difficult positions with variable anatomy and tissue resistance.
   
3. Reproducibility: Assesses the ability to stitch reliably and precisely all the time and every time. For rating this skill, the three levels are defined as 30–60%, 60–90% and more than 90% of the time.
   

Level 4 defines the ability to perform the surgical tasks with awareness of the surrounding environment including time and relations.

1. Pace: Assesses the ability to perform the task without hesitation, interruption, deviation or repetition. This emphasises economy of time and movement, so the operators at this level becomes attuned to their own natural rhythm. It is very important to stress that pace does not equate to speed.
   
2. Awareness: Assesses the ability to stay aware of time, surroundings and environment while performing the task. This aims to assess the ability of the trainee keep abreast of the whole situation including real time monitoring, while totally engaged in task.
   
3. Relations: Assesses the ability to communicate with the personnel around, which would represent the theatre team in real life, while performing the task.
   

The PAR-DOM gives a graphic representation of the surgical ability of the trainee. It highlights the deficiencies in needle skills and enables measurement of progression over time. Furthermore, the skills are distributed in the matrix so that progression can logically move diagonally from the left upper corner down to the right lower corner of the model and does not move along either the x or y axis alone. Therefore, with experience, we further defined the level expected from each of the trainees in each of six training years of any cardiothoracic programme along this diagonal progression (Fig. 2). This diagonal progression is deemed to be useful in assessing and planning further training, however, the skills are assessed concurrently and independently at the time of assessment.

View larger version (15K): [in this window] [in a new window]   Fig. 2. The progression of a surgical trainee through three successive assessments over a period of 8 months.

	3. Discussion

Top Abstract 1. Introduction 2. Materials and methods 3. Discussion References

The conventional apprenticeship model in surgical education does not satisfy the requirements of training at present nor in the future. It relies mainly on passive unreliable observation of live operating in theatre as a mode of both transferring knowledge by the trainer and its acquisition by the trainee [4]. Subjective assessment of the trainer in this situation is global and not based on previously set or studied criteria. However, this was still useful to give a limited assessment of the trainee's technical ability and progress and thus provide some feedback to trainees, but sadly far from being structured or sufficient to formulate a staged training plan according to the trainee's needs [5]. In fact, the subjective global rating scales by experts were still more reliable and valid than checklists alone in assessing the trainees' performance of procedures, and the addition of the checklists to these scales did not improve either validity or reliability [6]. Nevertheless, with this type of subjective assessment some trainees were progressing to become independent operators without demonstrating enough objective evidence of technical proficiency even at basic surgical skills level [7]. All this, in addition to the difficulties that surgical training has been facing lately [1, 2], has highlighted the urgent need for a structured objective assessment tool [5].

Patient safety during a surgical procedure remains a priority and a responsibility of the trainer to the extent that it could create a conflict between the ethical duty of the trainer towards patients and his or her societal commitment to train new surgeons [8]. Furthermore, training or assessment under the stress of the operating room environment is difficult and not an appealing experience for either the trainer or the trainee. Therefore, the operating room is not a place of training or assessment but a place of consolidation of an already gained and assessed training experience under very controlled and strict conditions of safety and clinical governance [9].

The duty of the trainer or the training body starts with the process of selection of the suitable trainable trainees in the first place, which does not seem to follow clear criteria at present. One realises that this selection is a very complicated and difficult process. However, it has been shown that the ability to be trained in surgical specialties does not rely solely on the academic knowledge but involves other non-cognitive, demographic factors and personality characteristics [10]. The creation of objective assessment tools may also help in this difficult selection process at an earlier stage.

Several assessment tools have been developed over the past two decades in order to help assess surgical ability and progress of the surgical trainees. The Objective Structured Assessment of Technical Skills (OSATS) has been widely used and validated. It consists of six bench-top stations, where the trainee's performance of a procedure or task is assessed by a checklist with a global rating scale [11]. These bench-top stations are usually set up tasks on cadaveric porcine tissue that differ according to the speciality of the trainee. The bench-top models have been dealing with full tasks, which can be modified according to the surgical speciality of the trainee. It could be an aortic cannulation for a cardiac trainee, but a ureteric anastomosis for a urology trainee. The emphasis is on the performance of the task as a whole but not on the fine needle skills that are required to perform this task [6].

Objective assessment of skills and consequently training should pay attention to the basic needle skills as they are the basic units that form the full procedure at the end. This attention should not only be aimed for as a fine tuning exercise in order to ‘polish the surgical technique of an already trained surgeon', but as an objective tool with which the technical ability of the surgeon is assessed and may be improved. It has been repeatedly stated that a ‘skilfully performed operation is 75% decision making and 25% dexterity’.

Dexterity analysis systems are able to assess this surgical handicraft up to a certain limit, however, they are still in the process of development and validation. When they are married with virtual reality they are able to assess and feedback about skill based errors [5]. Experience showed that assessment should marry training in order to get the benefit from both. Otherwise, it would be a useless exercise to diagnose trainees’ underperformances and not plan successful remedies for them. This highlights the ability to give and receive feedback in order to create a progression cycle to move the trainee from a certain level to a higher one.

The recent evidence suggests that there should be a differentiation between the performance profiles of a task and a skill in assessing surgical training. Future training strategies should aim to teach clearly defined surgical skills so the brain can build up the task from those skills every time it is required. It has been recently shown that surgeons were three times faster than non-trained medical students in performing a suturing task but not necessarily better [12].

The trainer (DOR) in this study, when diagnosed a technical deficiency, sent the trainee away with a handful of simple exercises to practise at home out of hours regularly. These exercises included performing needle skills on poached eggs, boiled potatoes and bananas and practising fine quick movements as rice picking. Others have thought and reported similar means of home based ‘out of hours’ continuous training on variable materials. Neurosurgeons could practise microvascular techniques on a chicken wing artery [13], anaethetists could practise on lemons and oranges to train on tissue resistance feedback for epidural insertion [14] and laparoscopic trainees could set an inexpensive ‘kitchen table’ trainer or simulator at home using a lab-top, a web-cam and a box [15].

We aimed to propose and describe the model in this publication, however, the next step would be to assess its validity, reliability and reproducibility. This model also provides a framework by which the trainees can assess their own ability and hence understand the specific skill that is limiting their progress along the diagonal progression.

	References

Top Abstract 1. Introduction 2. Materials and methods 3. Discussion References

 

1. Jackson B. Turbulent times – the experiences of a college president. Am J Surg 2003; 185:22–25.[CrossRef][Medline]
2. Chikwe J, de Souza A, Pepper J. No time to train the surgeons. Br Med J 2004; 328:418–419.[Free Full Text]
3. Hosseinpour A, Hilton G, Nashef S. Techniques of needle stance exchange in situ. Interact CardioVasc Thorac Surg 2005; 4:289–291.[Abstract/Free Full Text]
4. Naftalin A. Unfinished business and modernizing medical careers. Br Med J 2003; 326:S189–190.
5. Moorthy K, Munz Y, Sarker S, Darzi A. Objective assessment of technical skills in surgery. Br Med J 2003; 327:1032–1037.[Free Full Text]
6. Regehr G, MacRae H, Reznick RK, Szalay D. Comparing the psychometric properties of checklists and global rating scales for assessing performance on an OSCE-format examination. Acad Med 1998; 73:993–997.[Medline]
7. Hance J, Aggarwal R, Stanbridge R, Blauth C, Munz Y, Darzi A, Pepper J. Objective assessment of technical skills in cardiac surgery. Eur J Cardiothorac Surg 2003; 28:157–162.
8. Bernstein M. Surgical teaching: how should neurosurgeons handle the conflict of duty to today's patients with the duty to tomorrow's? Br J Neurosurg 2003; 17:121–123.[CrossRef][Medline]
9. Cox S, Swanson M. Identification of teaching excellence in operating room and clinic settings. Am J Surg 2002; 183:251–255.[CrossRef][Medline]
10. Tett R, Jackson D, Rothstien M. Personality measures as predictors of job performance: a meta-analytic review. Personnel Psychol 1991; 44:703–742.
11. Martin J, Regher G, Reznick R, MacRae H, Murnaghan J, Hutchison C, Brown M. Objective structured assessment of technical skill (OSATS) for surgical residents. Br J Surg 1997; 84:273–278.[CrossRef][Medline]
12. Pilitsis J, Redding G. The use of performance profiles in the design of a systematic medical suturing training programme. Appl Ergon 1977; 8:87–92.[CrossRef][Medline]
13. Hino A. Teaching in microvascular surgery using a chicken wing artery. Neurosurgery 2003; 52:1495–1498.[Medline]
14. Brown T. An orange or lemon as a teaching aid. Anaesth Intensive Care 2003; 31:334–335.[Medline]
15. Harris M, Hindmarsh J. Skills training and the role of skills centres. BJU Int 2003; 92:332.[Medline]

This article has been cited by other articles:

				S. C. Papaspyros, K. C. Javangula, and D. J. O'Regan Surgical training in the 48-h week: a novel simulation and educational tool. From amateur golfer to professional pilot Eur. J. Cardiothorac. Surg., September 1, 2009; 36(3): 511 - 515. [Abstract] [Full Text] [PDF]

This Article Abstract Full Text (PDF) On-line Video Alert me when this article is cited Alert me if a correction is posted Services Email this article to a friend Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Add to Personal Folders Download to citation manager Author home page(s): Sharif Al-Ruzzeh Shishir Karthik David O'Regan Permission Requests Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Al-Ruzzeh, S. Articles by O'Regan, D. Search for Related Content PubMed PubMed Citation Articles by Al-Ruzzeh, S. Articles by O'Regan, D. Related Collections Education Professional affairs