Frame-wise detection of surgeon stress levels during laparoscopic training using kinematic data

Yi Zheng, Grey Leonard, Herbert Zeh, Ann Majewicz Fey

Research output: Contribution to journalArticlepeer-review

4 Scopus citations


Purpose: Excessive stress experienced by the surgeon can have a negative effect on the surgeon’s technical skills. The goal of this study is to evaluate and validate a deep learning framework for real-time detection of stressed surgical movements using kinematic data. Methods: 30 medical students were recruited as the subjects to perform a modified peg transfer task and were randomized into two groups, a control group (n=15) and a stressed group (n=15) that completed the task under deteriorating, simulated stressful conditions. To classify stressed movements, we first developed an attention-based Long-Short-Term-Memory recurrent neural network (LSTM) trained to classify normal/stressed trials and obtain the contribution of each data frame to the stress level classification. Next, we extracted the important frames from each trial and used another LSTM network to implement the frame-wise classification of normal and stressed movements. Results: The classification between normal and stressed trials using attention-based LSTM model reached an overall accuracy of 75.86% under Leave-One-User-Out (LOUO) cross-validation. The second LSTM classifier was able to distinguish between the typical normal and stressed movement with an accuracy of 74.96% with an 8-second observation under LOUO. Finally, the normal and stressed movements in stressed trials could be classified with the accuracy of 68.18% with a 16-second observation under LOUO. Conclusion: In this study, we extracted the movements which are more likely to be affected by stress and validated the feasibility of using LSTM and kinematic data for frame-wise detection of stress level during laparoscopic training.

Original languageEnglish (US)
Pages (from-to)785-794
Number of pages10
JournalInternational Journal of Computer Assisted Radiology and Surgery
Issue number4
StatePublished - Apr 2022


  • Deep learning
  • Laparoscopic surgery
  • Motion analysis
  • Surgical stress

ASJC Scopus subject areas

  • Surgery
  • Biomedical Engineering
  • Radiology Nuclear Medicine and imaging
  • Computer Vision and Pattern Recognition
  • Computer Science Applications
  • Health Informatics
  • Computer Graphics and Computer-Aided Design


Dive into the research topics of 'Frame-wise detection of surgeon stress levels during laparoscopic training using kinematic data'. Together they form a unique fingerprint.

Cite this