Validity and Reliability of Emergency Severity Index and Conventional Three-Tier Triage System in the Emergency Department, Hospital Universiti Sains Malaysia

Background: The study aimed to examine the reliability and validity of the existing three-tier triaging system and a new five-level emergency triaging system, emergency severity index (ESI), in the Emergency Department (ED) of Hospital Universiti Sains Malaysia (HUSM).
Methods: This study was conducted in HUSM’s ED over two study periods. In the first three months, 300 patients were triaged under the three-tier triaging system, and, in the subsequent three months, 280 patients were triaged under the ESI. The patients were triaged by junior paramedics and the triage records were retained and later re-triaged by senior paramedics. The inter-rater reliability was evaluated using Cohen's Kappa statistics. The acuity ratings of the junior paramedics were compared with those of the expert panel to determine the sensitivity and specificity of each acuity level for both the ESI and the three-tier triaging system. The over-triage rate, under-triage rate, amount of resources used, admission rate and discharge rate were also determined.
Results: The inter-rater agreement for the three-tier triaging system was 0.81 while that of the ESI was 0.75. The ESI had a higher average sensitivity of 74.3% and a specificity of 94.4% while the three-tier system’s average sensitivity was 68.5% and its specificity 87.0%. The average under-triage and over-triage rates for the ESI were 10.7% and 6.2%, respectively, which were lower than the three-tier system’s average under-triage rate of 13.1% and over-triage rate of 17.1%. The urgency levels of both the ESI and the three-tier system were associated with increased admission rates and resources used in the ED.
Conclusion: The ESI’s inter-rater reliability was comparable to the three-tier triaging system and it demonstrated better validity than the existing three-tier system.

Keywords
emergency severity index; inter-rater agreement; sensitivity; specificity; triaging system