Steven Yule

BACKGROUND: Analyses of adverse events in surgery reveal that many underlying causes are behavioural, such as communication failure, rather than technical. Non-technical (i.e. cognitive and interpersonal) skills are not addressed explicitly in surgical training. However, surgeons need to demonstrate these skills, which underpin their technical excellence, to maximise patient safety in the operating theatre. This paper describes the method used to identify surgeons' non-technical skills, and the development of a skills taxonomy and behavioural rating system to structure observation and feedback in surgical training. METHODS: Cognitive task analyses (critical incident interviews) were conducted with 27 consultant surgeons in general, cardiac and orthopaedic surgery. The interviews were coded and a multidisciplinary group of surgeons and psychologists used an iterative process to develop a skills taxonomy. This was supported by data gathered from an attitude survey, literature review, analysis of surgical mortality reports and observations in theatre. RESULTS: Five categories of non-technical skills were identified, including situation awareness, decision making, task management, leadership and communication and teamwork. This provided a structure for a prototype skill taxonomy (v1.1), which comprised 14 non-technical skill elements. Observable behaviours (markers) indicative of good and poor performance were developed for each element by 16 consultant surgeons to form a prototype behaviour rating system. CONCLUSIONS: The prototype skills taxonomy and behaviour rating system are grounded empirically in surgery. The reliability of the system is currently being tested using standardised scenarios. If this evaluation proves successful, the system could be used to structure feedback and guide non-technical skills training.

BACKGROUND: Previous research has shown that surgeons' intraoperative non-technical skills are related to surgical outcomes. The aim of this study was to evaluate the reliability of the NOTSS (Non-technical Skills for Surgeons) behavior rating system. Based on task analysis, the system incorporates five categories of skills for safe surgical practice (Situation Awareness, Decision Making, Task Management, Communication & Teamwork, and Leadership). METHODS: Consultant (attending) surgeons (n = 44) from five Scottish hospitals attended one of six experimental sessions and were trained to use the NOTSS system. They then used the system to rate consultant surgeons' behaviors in six simulated operating room scenarios that were presented using video. Surgeons' ratings of the behaviors demonstrated in each scenario were compared to expert ratings ("accuracy"), and assessed for inter-rater reliability and internal consistency. RESULTS: The NOTSS system had a consistent internal structure. Although raters had minimal training, rating "accuracy" for acceptable/unacceptable behavior was above 60% for all categories, with mean of 0.67 scale points difference from reference (expert) ratings (on 4-point scale). For inter-rater reliability, the mean values of within-group agreement (r (wg)) were acceptable for the categories Communication & Teamwork (.70), and Leadership (.72), but below a priori criteria for other categories. Intra-class correlation coefficients (ICC) indicated high agreement using average measures (values were .95-.99). CONCLUSIONS: With the requisite training, the prototype NOTSS system could be used reliably by surgeons to observe and rate surgeons' behaviors. The instrument should now be tested for usability in the operating room.

BACKGROUND: Surgeons in the operating theatre deal constantly with high-demand tasks that require simultaneous processing of a large amount of information. In certain situations, high cognitive load occurs, which may impact negatively on a surgeon's performance. This systematic review aims to provide a comprehensive understanding of the different methods used to assess surgeons' cognitive load, and a critique of the reliability and validity of current assessment metrics. METHODS: A search strategy encompassing MEDLINE, Embase, Web of Science, PsycINFO, ACM Digital Library, IEEE Xplore, PROSPERO and the Cochrane database was developed to identify peer-reviewed articles published from inception to November 2016. Quality was assessed by using the Medical Education Research Study Quality Instrument (MERSQI). A summary table was created to describe study design, setting, specialty, participants, cognitive load measures and MERSQI score. RESULTS: Of 391 articles retrieved, 84 met the inclusion criteria, totalling 2053 unique participants. Most studies were carried out in a simulated setting (59 studies, 70 per cent). Sixty studies (71 per cent) used self-reporting methods, of which the NASA Task Load Index (NASA-TLX) was the most commonly applied tool (44 studies, 52 per cent). Heart rate variability analysis was the most used real-time method (11 studies, 13 per cent). CONCLUSION: Self-report instruments are valuable when the aim is to assess the overall cognitive load in different surgical procedures and assess learning curves within competence-based surgical education. When the aim is to assess cognitive load related to specific operative stages, real-time tools should be used, as they allow capture of cognitive load fluctuation. A combination of both subjective and objective methods might provide optimal measurement of surgeons' cognition.