WHO Surgical Safety Checklist: Complications and Death Before and After Implementation in Eight Hospitals
The WHO Surgical Safety Checklist study was a prospective before-and-after study of 3,733 patients before and 3,955 after introduction of a 19-item checklist, read aloud by the operating team at sign in, time out and sign out, in patients aged 16 or older having noncardiac surgery at eight hospitals in eight cities. In-hospital major complications, including death, within 30 days were 11.0% before and 7.0% after, and death was 1.5% before and 0.8% after.[1]
- Design and population
- Prospective before-and-after study at eight hospitals. Consecutive patients aged 16 or older undergoing noncardiac surgery in designated operating rooms of eight hospitals in eight cities, four in high-income and four in low- or middle-income countries.
- Arms
- Care after introduction of the 19-item WHO Surgical Safety Checklist versus care before introduction of the checklist
- Report and follow-up
- 2009 · In hospital until discharge or 30 days, whichever came first
| Endpoint | Results | Effect | Finding |
|---|---|---|---|
| In-hospital rate of any major complication, including death, within 30 days (site-standardized) | 7.0% vs 11.0% | difference -4; P <0.001 | Met |
| In-hospital death within 30 days (site-standardized) | 0.8% vs 1.5% | difference -0.7; P 0.003 | Met |
| Patients receiving all six measured safety processes (N = 7,688) | 56.7% vs 34.2% | P <0.001 | Descriptive |
| Antibiotics within 60 minutes before incision when indicated (N = 6,802) | 82.6% vs 56.1% | P <0.001 | Descriptive |
| Oral confirmation immediately before incision (N = 7,688) | 92.3% vs 54.4% | P <0.001 | Descriptive |
| Sponge count completed when an incision was made (N = 7,572) | 94.6% vs 84.6% | P <0.001 | Descriptive |
Harms: Surgical-site infection: 3.4% vs 6.2%; Unplanned return to the operating room: 1.8% vs 2.4%; Pneumonia: 1.3% vs 1.1%
Registry not recorded · Report DOI
Design
The eight pilot sites of the World Health Organization's Safe Surgery Saves Lives program were chosen for geographic distribution across WHO regions and for a range of economic settings: four in high-income and four in low- or middle-income countries. Each hospital named a coinvestigator to lead the project, had the support of its administration, and designated one to four study operating rooms. A full-time local data collector without clinical duties was trained by the primary investigators to record complications and process measures. Consecutive patients aged 16 or older having noncardiac surgery in those rooms were enrolled between October 2007 and September 2008, aiming for 500 per site in each phase; the sample size was calculated to detect a 20% reduction in complications with 80% power at an alpha of 0.05. None of the hospitals had a formal pre- or postoperative team briefing or a standard plan for intravenous access in cases of high blood loss before the study. The World Health Organization funded the study.[1]
The primary endpoint was any major complication, including death, during the postoperative hospital stay, followed until discharge or day 30, whichever came first. Complications followed the American College of Surgeons NSQIP definitions: acute renal failure, transfusion of 4 or more units of red cells within 72 hours, cardiac arrest requiring resuscitation, coma of 24 hours or longer, deep-vein thrombosis, myocardial infarction, unplanned intubation, ventilation for 48 hours or longer, pneumonia, pulmonary embolism, stroke, major wound disruption, surgical-site infection, sepsis, septic shock, the systemic inflammatory response syndrome, unplanned return to the operating room, vascular graft failure, and death. Urinary tract infection was not counted as a major complication. Physician reviewers decided by consensus whether events recorded as other complications qualified, using the Clavien classification for guidance.[1]
Rates were standardized to each site's share of patients, and P values came from logistic regression with site as a fixed effect and generalized estimating equations to test for clustering. No adjustment was made for multiple comparisons. Robustness analyses added the presence of a data collector in the operating room and the case mix (orthopedic, thoracic, nonobstetric abdominopelvic, obstetric, vascular, endoscopic or other), removed each site in turn, and split the sites by income group.[1]
Results
Major complications fell from 11.0% to 7.0% (P < 0.001), a fall of 36% on average, and in-hospital death fell from 1.5% to 0.8% (P = 0.003). Every site had fewer complications; the reduction was significant at three sites, one high-income and two lower-income.[1] In a later analysis restricted to 1,750 patients having urgent operations, defined as surgery needed within 24 hours of assessment, complications including death fell from 18.4% to 11.7% (P = 0.0001) and death from 3.7% to 1.4% (P = 0.0067).[2]
The complication result remained significant when each site was removed in turn (P < 0.05), after adding data-collector presence or case mix to the model (P < 0.001 for both), and with clustering by site (P = 0.001). Complications fell from 10.3% to 7.1% at the high-income sites and from 11.7% to 6.8% at the lower-income sites (P < 0.001 for each). Death fell from 0.9% to 0.6% at high-income sites (P = 0.18) and from 2.1% to 1.0% at lower-income sites (P = 0.006), so only the lower-income difference was significant.[1]
Case mix was similar across phases: mean age 46.8 and 46.7 years, female sex 56.2% and 57.6%, urgent cases 22.3% and 23.3%, outpatient procedures 9.9% and 9.4%, and general anesthesia 77.0% and 77.3% (before and after; none significantly different).[1]
Surgery and treatment delivery
The checklist is an oral confirmation by the operating team at three points. Before induction, at least the nurse and an anesthesia professional confirm identity, site, procedure and consent, that the surgical site is marked, that the pulse oximeter is working, allergy status, airway and aspiration risk, and access and fluids when blood loss of 500 mL or more (or 7 mL/kg in children) is possible. Before skin incision, the whole team introduces itself by name and role, confirms the patient, site and procedure, reviews anticipated critical events (the surgeon reviews critical or unexpected steps, duration and expected blood loss), confirms antibiotic prophylaxis within 60 minutes before incision unless not indicated, and confirms that essential imaging is displayed. Before the patient leaves the room, the nurse reviews the recorded procedure name, needle, sponge and instrument counts, specimen labeling and equipment problems, and the surgeon, nurse and anesthesia professional state the key concerns for recovery.[1]
Each site introduced the checklist to its study rooms over 1 week to 1 month, using local lectures, written materials or direct guidance, after receiving a summary of its baseline deficiencies. The primary investigators also sent a recorded video, held a teleconference with each local team and visited each site. Three institutions had to change where prophylactic antibiotics were given. Only pulse oximetry and prophylactic antibiotics required significant resources, and both were already available at every site, including the lower-income hospitals, although inconsistently used.[1]
Measured processes improved. Patients who received all six measured safety steps rose from 34.2% to 56.7% (P < 0.001). Antibiotics within 60 minutes before incision when indicated rose from 56.1% to 82.6%, oral confirmation of patient identity and operative site immediately before incision from 54.4% to 92.3%, and a completed sponge count when an incision was made (N = 7,572) from 84.6% to 94.6% (P < 0.001 for each). Adherence to the team introductions, briefings and debriefings could not be measured.[1]
Adverse events
Surgical-site infection fell from 6.2% to 3.4% (P < 0.001), with the largest change at one lower-income site, where it fell from 20.5% to 3.6%. Unplanned return to the operating room fell from 2.4% to 1.8% (P = 0.047). Pneumonia was 1.1% before and 1.3% after (P = 0.46).[1]
Limitations
Consecutive patients were compared before and after implementation in the same rooms because randomizing the checklist to rooms risked cross-contamination, so there is no concurrent control group. The authors limited the study to under 1 year to reduce secular trends and cite an NSQIP review of 2007 US data that showed no substantial change in death or complications. A Hawthorne effect cannot be excluded; adding the presence of a study data collector, who directly observed 37.5% of patients, did not change the significance of the complication result. Complications were collected only during the hospital stay, so follow-up of outpatient procedures ended at same-day discharge and their complications are likely underestimated. Data collectors were trained at the start, and a learning curve could have affected capture; the authors judge that this would have lowered the measured effect. The sample size was calculated for complications rather than death. Team briefings and system changes were introduced together with the checklist items, so the study cannot separate the contribution of individual items.[1]
In conversation
Later studies used different designs. SURPASS, a broader checklist spanning the whole surgical pathway, was studied in six hospitals with high standards of care and five control hospitals; complications per 100 patients fell from 27.3 to 16.7 and in-hospital mortality from 1.5% to 0.8%, with no change in the control hospitals.[3] A stepped-wedge cluster randomized trial in two hospitals across five surgical clusters, including urologic surgery, found complications falling from 19.9% to 11.5% in 4,475 procedures and mortality falling from 1.6% to 1.0%, a reduction that was not significant overall.[4] An Ontario analysis of administrative data from 101 hospitals, under a provincial policy encouraging universal checklist adoption, reported adjusted operative mortality of 0.71% before and 0.65% after (odds ratio 0.91, 95% CI 0.80 to 1.03) and adjusted complications of 3.86% before and 3.82% after (odds ratio 0.97, 95% CI 0.90 to 1.03), with no significant reduction in either.[5]
Guidelines and approvals
- WHO (Implementation Manual, Safe Surgery Saves Lives, 2009). The manual presents the checklist as a tool for clinicians, states that its use has been demonstrably associated with significant reductions in complication and death rates in diverse hospitals (citing this study), and requires that a single person be made responsible for performing the checks, often a circulating nurse. The checklist form states that additions and modifications to fit local practice are encouraged.[6]
Report history
| Published | Report | Cutoff | Headline results | Citation |
|---|---|---|---|---|
| 2009-01-14 | Initial report · primary | — | In-hospital rate of any major complication, including death, within 30 days (site-standardized): 7.0% vs 11.0%; difference -4; P <0.001 Met In-hospital death within 30 days (site-standardized): 0.8% vs 1.5%; difference -0.7; P 0.003 Met Major complications, four high-income sites: 7.1% vs 10.3%; P <0.001 Major complications, four low- or middle-income sites: 6.8% vs 11.7%; P <0.001 Death, four high-income sites: 0.6% vs 0.9%; P 0.18 Death, four low- or middle-income sites: 1.0% vs 2.1%; P 0.006 Patients receiving all six measured safety processes (N = 7,688): 56.7% vs 34.2%; P <0.001 Descriptive Antibiotics within 60 minutes before incision when indicated (N = 6,802): 82.6% vs 56.1%; P <0.001 Descriptive Oral confirmation immediately before incision (N = 7,688): 92.3% vs 54.4%; P <0.001 Descriptive Sponge count completed when an incision was made (N = 7,572): 94.6% vs 84.6%; P <0.001 Descriptive | Haynes AB, Weiser TG, Berry WR, et al. "A Surgical Safety Checklist to Reduce Morbidity and Mortality in a Global Population." *N Engl J Med.* 2009;360(5):491-499. doi:10.1056/NEJMsa0810119 DOI |
| 2010 | Urgent operations · secondary | — | Complication rate including death, in-hospital up to 30 days: 11.7% vs 18.4%; difference -6.7; P 0.0001 Death rate, in-hospital up to 30 days: 1.4% vs 3.7%; difference -2.3; P 0.0067 Adherence to six measured safety steps: 50.7% vs 18.6%; P <0.0001 Descriptive | Weiser TG, Haynes AB, Dziekan G, et al. "Effect of a 19-item Surgical Safety Checklist During Urgent Operations in a Global Patient Population." *Ann Surg.* 2010;251(5):976-980. doi:10.1097/SLA.0b013e3181d970e3 DOI |
References
1. Haynes AB, Weiser TG, Berry WR, et al. "A Surgical Safety Checklist to Reduce Morbidity and Mortality in a Global Population." N Engl J Med. 2009;360(5):491-499. doi:10.1056/NEJMsa0810119
2. Weiser TG, Haynes AB, Dziekan G, et al. "Effect of a 19-item Surgical Safety Checklist During Urgent Operations in a Global Patient Population." Ann Surg. 2010;251(5):976-980. doi:10.1097/SLA.0b013e3181d970e3
3. de Vries EN, Prins HA, Crolla RM, et al. "Effect of a Comprehensive Surgical Safety System on Patient Outcomes." N Engl J Med. 2010;363(20):1928-1937. doi:10.1056/NEJMsa0911535
4. Haugen AS, Søfteland E, Almeland SK, et al. "Effect of the World Health Organization Checklist on Patient Outcomes: A Stepped Wedge Cluster Randomized Controlled Trial." Ann Surg. 2015;261(5):821-828. doi:10.1097/SLA.0000000000000716
5. Urbach DR, Govindarajan A, Saskin R, et al. "Introduction of Surgical Safety Checklists in Ontario, Canada." N Engl J Med. 2014;370(11):1029-1038. doi:10.1056/NEJMsa1308261
6. World Health Organization. Implementation Manual: WHO Surgical Safety Checklist 2009. Safe Surgery Saves Lives. Geneva: World Health Organization; 2009. ISBN 978 92 4 159859 0.