AI-Empowered Anticipation of Surgical Triplets
in Laparoscopic Videos for Enhanced Decision Support
Currently Anonymous

Abstract
The development of intra-operative context-aware decision support systems is crucial for enhancing decision-making and analysing surgical workflow. We introduce SurgAnt, a diffusion-based model designed to aid decision-making during surgery by predicting action triplets several frames ahead based on laparoscopic video footage, where each action triplet is composed by: <Instrument, Verb, Target>.
We evaluated our model on laparoscopic videos performed by professional surgeons. The results demonstrated that our model can accurately predict the next triplets in most cases, highlighting its potential to assist operating surgeons during surgery. Our model can also serve as the training tool for medical interns and can be integrated into other computer-assisted intervention (CAI) systems.

Technical Overview

Surgical Triplets

Module Structure

DEMO

Description for Frame 1

Description for Frame 2