Abstract

Cricothyrotomy serves as one of the most efficient surgical interventions when a patient is enduring a can't intubate can't oxygenate (CICO) scenario. However, medical background and professional training are required for the provider to establish a patent airway successfully. Motivated by robotics applications in search and rescue, this work focuses on applying artificial intelligence techniques to the precise localization of the incision site, the cricothyroid membrane (CTM), of the injured using an RGB-D camera, and the manipulation of a robot arm with reinforcement learning to reach the detected CTM keypoint. In this paper, we proposed a deep learning-based model, the hybrid neural network (HNNet), to detect the CTM with a success rate of 96.6%, yielding an error of less than 5 mm in real-world coordinates. In addition, a separate neural network was trained to manipulate a robotic arm for reaching a waypoint with an error of less than 5 mm. An integrated system that combines both the perception and the control techniques was built and experimentally validated using a human-size manikin to prove the overall concept of autonomous cricothyrotomy with an RGB-D camera and a robotic manipulator using artificial intelligence.

References

1.
Macdonald
,
J. C.
, and
Tien
,
H. C.
,
2008
, “
Emergency Battlefield Cricothyrotomy
,”
Can. Med. Assoc. J.
,
178
(
9
), pp.
1133
1135
.10.1503/cmaj.080036
2.
Pracy
,
J.
,
Brennan
,
L.
,
Cook
,
T.
,
Hartle
,
A.
,
Marks
,
R.
,
McGrath
,
B.
,
Narula
,
A.
, and
Patel
,
A.
,
2016
, “
Surgical Intervention During a Can't Intubate Can't Oxygenate (CICO) Event: Emergency Front-of-Neck Airway (FONA)?
,”
Br. J. Anaesth.
,
117
(
4
), pp.
426
428
.10.1093/bja/aew221
3.
Meyer
,
T.
, and
Patel
,
S.
,
2014
, “
Surgical Airway
,”
Int. J. Crit. Illn. Injury Sci.
,
4
(
1
), p.
71
.10.4103/2229-5151.128016
4.
Aslani
,
A.
,
Ng
,
S. C.
,
Hurley
,
M.
,
McCarthy
,
K. F.
,
McNicholas
,
M.
, and
McCaul
,
C. L.
,
2012
, “
Accuracy of Identification of the Cricothyroid Membrane in Female Subjects Using Palpation: An Observational Study
,”
Anesth. Analg.
,
114
(
5
), pp.
987
992
.10.1213/ANE.0b013e31824970ba
5.
Katos
,
M. G.
, and
Goldenberg
,
D.
,
2007
, “
Emergency Cricothyrotomy
,”
Oper. Tech. Otolaryngol. Head Neck Surg.
,
18
(
2
), pp.
110
114
.10.1016/j.otot.2007.05.002
6.
Lind
,
M. M.
,
Corridore
,
M.
,
Sheehan
,
C.
,
Moore-Clingenpeel
,
M.
, and
Maa
,
T.
,
2018
, “
A Multidisciplinary Approach to a Pediatric Difficult Airway Simulation Course
,”
Otolaryngol. Head Neck Surg.
,
159
(
1
), pp.
127
135
.10.1177/0194599818758993
7.
Williams
,
A.
,
Sebastian
,
B.
, and
Ben-Tzvi
,
P.
,
2019
, “
Review and Analysis of Search, Extraction, Evacuation, and Medical Field Treatment Robots
,”
J. Intell. Rob. Syst. Theory Appl.
,
96
(
3–4
), pp.
401
418
.10.1007/s10846-019-00991-6
8.
Marescaux
,
J.
,
Leroy
,
J.
,
Gagner
,
M.
,
Rubino
,
F.
,
Mutter
,
D.
,
Vix
,
M.
,
Butner
,
S. E.
, and
Smith
,
M. K.
,
2001
, “
Transatlantic Robot-Assisted Telesurgery
,”
Nature
,
413
(
6854
), pp.
379
380
.10.1038/35096636
9.
Chapman
,
P. L.
,
Cabrera
,
L. D.
,
Varela-Mayer
,
C.
,
Baker
,
M. M.
,
Elnitsky
,
C.
,
Figley
,
C.
,
Thurman
,
R. M.
,
Lin
,
C.-D.
, and
Mayer
,
L. P.
,
2012
, “
Training, Deployment Preparation, and Combat Experiences of Deployed Health Care Personnel: Key Findings From Deployed U.S. Army Combat Medics Assigned to Line Units
,”
Mil. Med.
,
177
(
3
), pp.
270
277
.10.7205/MILMED-D-11-00305
10.
Cardenas
,
A.
,
Quiroz
,
O.
,
Hernández
,
R.
,
Medellin-Castillo
,
H. I.
,
González
,
A.
,
Maya
,
M.
, and
Piovesan
,
D.
,
2021
, “
Vision-Based Control of a Mobile Manipulator With an Adaptable-Passive Suspension for Unstructured Environments
,”
ASME J. Mech. Rob.
,
13
(
5
), p.
050908
.10.1115/1.4051437
11.
Johnson
,
B. V.
,
Gong
,
Z.
,
Cole
,
B. A.
, and
Cappelleri
,
D. J.
,
2019
, “
Design of Compliant Three-Dimensional Printed Surgical End-Effectors for Robotic Lumbar Discectomy
,”
ASME J. Mech. Rob.
,
11
(
2
), p.
020914
.10.1115/1.4042543
12.
Cao
,
Z.
,
Simon
,
T.
,
Wei
,
S.-E.
, and
Sheikh
,
Y.
,
2017
, “
Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields
,”
2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
,
Honolulu, HI
, July 22–25, Vol.
2017
-January, pp.
1302
1310
.
13.
Ren
,
S.
,
He
,
K.
,
Girshick
,
R.
, and
Sun
,
J.
,
2017
, “
Faster R-CNN: Towards Real-Time Object Detection With Region Proposal Networks
,”
IEEE Trans. Pattern Anal. Mach. Intell.
,
39
(
6
), pp.
1137
1149
.10.1109/TPAMI.2016.2577031
14.
Han
,
X.
,
Ren
,
H.
, and
Ben-Tzvi
,
P.
,
2020
, “
Autonomous Cricothyroid Membrane Detection Using Neural Networks for First-Aid Surgical Airway Management
,”
International Design Engineering Technical Conferences
, Online, Aug. 17–19.10.1115/DETC2020-22190
15.
Ren
,
T.
,
Dong
,
Y.
,
Wu
,
D.
, and
Chen
,
K.
,
2018
, “
Learning-Based Variable Compliance Control for Robotic Assembly
,”
ASME J. Mech. Rob.
,
10
(
6
), p.
061008
.10.1115/1.4041331
16.
Zhang
,
Z.
,
2012
, “
Microsoft Kinect Sensor and Its Effect
,”
IEEE Multimed.
,
19
(
2
), pp.
4
10
.10.1109/MMUL.2012.24
17.
Simonyan
,
K.
, and
Zisserman
,
A.
,
2015
, “
Very Deep Convolutional Networks for Large-Scale Image Recognition
,”
3rd International Conference on Learning Representations, ICLR 2015
, Conference Track Proceedings,
San Diego, CA
, May 7–9,
p. 1409.1556.https://ui.adsabs.harvard.edu/abs/2014arXiv1409.1556S/abstract
18.
Choi
,
P. J.
,
Oskouian
,
R. J.
,
Tubbs
,
R. S.
,
Xu
,
S.
,
Perez
,
M.
,
Yang
,
K.
,
Perrenot
,
C.
,
Felblinger
,
J.
,
Hubert
,
J.
,
Butner
,
S.
,
Ghodoussi
,
M.
,
Wu
,
B.
,
Wan
,
A.
,
Yue
,
X.
,
Jin
,
P.
,
Zhao
,
S.
,
Golmant
,
N.
,
Gholaminejad
,
A.
,
Gonzalez
,
J.
, and
Keutzer
,
K.
,
2018
, “
Transforming a Surgical Robot for Human Telesurgery
,”
Surg. Endosc.
,
28
(
5
), pp.
69
78
.10.1109/TRA.2003.817214
19.
Todorov
,
E.
,
Erez
,
T.
, and
Tassa
,
Y.
,
2012
, “
MuJoCo: A Physics Engine for Model-Based Control
,”
2012 IEEE/RSJ International Conference on Intelligent Robots and Systems
,
Algarve, Portugal
, Oct. 7–12, pp.
5026
5033
.10.1109/IROS.2012.6386109
20.
Ren
,
H.
, and
Ben-Tzvi
,
P.
,
2020
, “
Advising Reinforcement Learning Toward Scaling Agents in Continuous Control Environments With Sparse Rewards
,”
Eng. Appl. Artif. Intell.
,
90
, p.
103515
.10.1016/j.engappai.2020.103515
21.
Andrychowicz
,
M.
,
Wolski
,
F.
,
Ray
,
A.
,
Schneider
,
J.
,
Fong
,
R.
,
Welinder
,
P.
,
McGrew
,
B.
,
Tobin
,
J.
,
Pieter Abbeel
,
O.
, and
Zaremba
,
W.
,
2017
, “
Hindsight Experience Replay
,” 31st Conference on Neural Information Processing Systems (
NIPS 2017
),
Long Beach, CA
, Dec. 4–9, Vol.
30
, p. 1707.01495.https://arxiv.org/abs/1707.01495
22.
Lillicrap
,
T. P.
,
Hunt
,
J. J.
,
Pritzel
,
A.
,
Heess
,
N.
,
Erez
,
T.
,
Tassa
,
Y.
,
Silver
,
D.
, and
Wierstra
,
D.
,
2016
, “
Continuous Control With Deep Reinforcement Learning
,”
4th International Conference on Learning Representations, ICLR 2016
, Conference Track Proceedings,
San Juan, Puerto Rico,
May 2–4,
p. 1509.02971.https://arxiv.org/abs/1509.02971
23.
Li
,
W.
,
Dong
,
M.
,
Lu
,
N.
,
Lou
,
X.
, and
Sun
,
P.
,
2018
, “
Simultaneous Robot–World and Hand–Eye Calibration Without a Calibration Object
,”
Sensors (Basel, Switzerland)
,
18
(
11
), p.
3949
.10.3390/s18113949
You do not currently have access to this content.