Abstract
Measuring the motions of human hand joints is often a challenge due to the high number of degrees-of-freedom. In this study, we proposed a hand tracking system utilizing action cameras and ArUco markers to continuously measure the rotation angles of hand joints during motion. Three methods were developed to estimate the joint rotation angles. The pos-based method transforms marker positions to a reference coordinate system and extracts a hand skeleton to identify the rotation angles. Similarly, the orient-x-based method calculates the rotation angles from the transformed x-orientations of the detected markers in the reference coordinate system. In contrast, the orient-mat-based method first identifies the rotation angles in each camera coordinate system using the detected orientations and then synthesizes the results regarding each joint. Experiment results indicated that the repeatability errors with one camera regarding different marker sizes were around 2.64–27.56 deg and 0.60–2.36 deg using the marker positions and orientations, respectively. With multiple cameras employed, the joint rotation angles measured by using the three methods were compared with that measured by a goniometer. Comparison results indicated that the results of using the orient-mat-based method are more stable and efficient and can describe more types of movements. The effectiveness of this method was further verified by capturing hand movements of several participants. Therefore, it is recommended for measuring joint rotation angles in practical setups.
Issue Section:
Research Papers
Topics:
Rotation
References
1.
ASSH
, 2021
, “Body Anatomy: Upper Extremity Joints | The Hand Society
,” https://www.assh.org/handcareprod/safety/joints, Accessed June 15, 2021.2.
Bullock
, I. M.
, Júlia
, B.
, and Aaron
, M. D.
, 2012
, “Assessing Assumptions in Kinematic Hand Models: A Review
,” 4th IEEE RAS & EMBS International Conference on Biomedical Robotics and Biomechatronics (BioRob)
, Rome, Italy
, June 24–27
, pp. 139
–146
. 3.
Kuch
, J. J.
, and Huang
, T. S.
, 1994
, “Human Computer Interaction via the Human Hand: A Hand Model
,” Proceedings of 1994 28th Asilomar Conference on Signals, Systems and Computers
, Pacific Grove, CA
, Oct. 31–Nov. 2
, pp. 1252
–1256
.4.
Reissner
, L.
, Gabriella
, F.
, Renate
, L.
, Pietro
, G.
, and Maurizio
, C.
, 2019
, “Assessment of Hand Function During Activities of Daily Living Using Motion Tracking Cameras: A Systematic Review
,” Proc. Inst. Mech. Eng. Part H: J. Eng. Med.
, 233
(8
), pp. 764
–783
. 5.
Yang
, Y.
, Yuan
, T.
, Huysmans
, T.
, Elkhuizen
, W. S.
, Tajdari
, F.
, and Song
, Y.
, 2021
, “Posture-Invariant Three Dimensional Human Hand Statistical Shape Model
,” ASME J. Comput. Inf. Sci. Eng.
, 21
(3
), p. 031006
. 6.
Yang
, Y.
, Xu
, J.
, Elkhuizen
, W. S.
, and Song
, Y.
, 2021
, “The Development of a Low-Cost Photogrammetry-Based 3D Hand Scanner
,” HardwareX
, 10
, p. e00212
. 7.
Buckingham
, G.
, 2021
, “Hand Tracking for Immersive Virtual Reality: Opportunities and Challenges
,” arXiv preprint
, https://arxiv.org/abs/2103.14853Accessed November 15, 2021.8.
Hsiao
, J. H.
, Deng
, Y. H.
, Pao
, T. Y.
, Chou
, H. R.
, and Chang
, J. Y.
, 2017
, “Design of a Wireless 3d Hand Motion Tracking and Gesture Recognition Glove for Virtual Reality Applications
,” Proceedings of ISPS
, San Francisco, CA
, Aug. 29–30
, ASME Paper No. ISPS2017-5450.9.
Thakur
, A.
, and Rai
, R.
, “User Study of Hand Gestures for Gesture Based 3D CAD Modeling
,” Proceedings of IDETC/CIE
, Boston, MA
, Aug. 2–5
, ASME Paper No. DETC2015-46086.10.
Bullock
, I. M.
, Ma
, R. R.
, and Dollar
, A. M.
, 2013
, “A Hand-Centric Classification of Human and Robot Dexterous Manipulation
,” IEEE Trans. Haptics
, 6
(2
), pp. 129
–144
. 11.
Wang
, J.
, Barry
, O.
, Kurdila
, A. J.
, and Vijayan
, S.
, “On the Dynamics and Control of a Full Wrist Exoskeleton for Tremor Alleviation
,” Dynamic Systems and Control Conference
, Part City, UT
, Oct. 8–11
.12.
Ten Kate
, J.
, Smit
, G.
, and Breedveld
, P.
, 2017
, “3D-Printed Upper Limb Prostheses: A Review
,” Disabil. Rehabil. Assist. Technol.
, 12
(3
), pp. 300
–314
. 13.
Hamilton
, G. F.
, and Lachenbruch
, P. A.
, 1969
, “Reliability of Goniometers in Assessing Finger Joint Angle
,” Phys. Ther.
, 49
(5
), pp. 465
–469
. 14.
Reissner
, L.
, Fischer
, G.
, List
, R.
, Taylor
, W. R.
, Giovanoli
, P.
, and Calcagni
, M.
, 2019
, “Minimal Detectable Difference of the Finger and Wrist Range of Motion: Comparison of Goniometry and 3D Motion Analysis
,” J. Orthop. Surg. Res.
, 14
(1
), pp. 1
–10
. 15.
Ellis
, B.
, and Bruton
, A.
, 2002
, “A Study to Compare the Reliability of Composite Finger Flexion With Goniometry for Measurement of Range of Motion in the Hand
,” Clin. Rehabil.
, 16
(5
), pp. 562
–570
. 16.
Lewis
, E.
, Fors
, L.
, and Tharion
, W. J.
, 2010
, “Interrater and Intrarater Reliability of Finger Goniometric Measurements
,” Am. J. Occup. Ther.
, 64
(4
), pp. 555
–561
. 17.
Erol
, A.
, Bebis
, G.
, Nicolescu
, M.
, Boyle
, R. D.
, and Twombly
, X.
, 2007
, “Vision-Based Hand Pose Estimation: A Review
,” Comput. Vis. Image Understand.
, 108
(1–2
), pp. 52
–73
. 18.
Zhou
, H.
, and Hu
, H.
, 2008
, “Human Motion Tracking for Rehabilitation—A Survey
,” Biomed. Signal Process. Control
, 3
(1
), pp. 1
–18
. 19.
Chen
, W.
, Yu
, C.
, Tu
, C.
, Lyu
, Z.
, Tang
, J.
, Ou
, S.
, Fu
, Y.
, and Xue
, Z.
, 2020
, “A Survey on Hand Pose Estimation With Wearable Sensors and Computer-Vision-Based Methods
,” Sensors
, 20
(4
), p. 1074
. 20.
Yang
, H.
, Shao
, L.
, Zheng
, F.
, Wang
, L.
, and Song
, Z.
, 2011
, “Recent Advances and Trends in Visual Tracking: A Review
,” Neurocomputing
, 74
(18
), pp. 3823
–3831
. 21.
Connolly
, J.
, Condell
, J.
, O’Flynn
, B.
, Sanchez
, J. T.
, and Gardiner
, P.
, 2017
, “IMU Sensor-Based Electronic Goniometric Glove for Clinical Finger Movement Analysis
,” IEEE Sens. J.
, 18
(3
), pp. 1273
–1281
. 22.
Salchow-Hömmen
, C.
, Callies
, L.
, Laidig
, D.
, Valtin
, M.
, Schauer
, T.
, and Seel
, T.
, 2019
, “A Tangible Solution for Hand Motion Tracking in Clinical Applications
,” Sensors
, 19
(1
), p. 208
. 23.
Lin
, B. S.
, Lee
, I.
, Yang
, S. Y.
, Lo
, Y. C.
, Lee
, J.
, and Chen
, J. L.
, 2018
, “Design of an Inertial-Sensor-Based Data Glove for Hand Function Evaluation
,” Sensors
, 18
(5
), p. 1545
. 24.
Baldi
, T. L.
, Mohammadi
, M.
, Scheggi
, S.
, and Prattichizzo
, D.
, 2015
, “Using Inertial and Magnetic Sensors for Hand Tracking and Rendering in Wearable Haptics
,” Proceedings of IEEE World Haptics Conference (WHC)
, IL
, June 22–26
, pp. 381
–387
.25.
NDI
, “3D Guidance Sensors—NDI
,” https://www.ndigital.com/products/3d-guidance/3d-guidance-sensors/, Accessed August 10, 2021.26.
Saggio
, G.
, Riillo
, F.
, Sbernini
, L.
, and Quitadamo
, L. R.
, 2015
, “Resistive Flex Sensors: A Survey
,” Smart Mater. Struct.
, 25
(1
), p. 013001
. 27.
Michaud
, H. O.
, Dejace
, L.
, De Mulatier
, S.
, and Lacour
, S. P.
, “Design and Functional Evaluation of an Epidermal Strain Sensing System for Hand Tracking
,” Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
, Daejeon, South Korea
, Oct. 9–14
, pp. 3186
–3191
.28.
CyberGlove Systems LLC
, “CyberGlove III—CyberGlove Systems LLC
,” http://www.cyberglovesystems.com/cyberglove-iii/, Accessed August 10, 2021.29.
Jarque-Bou
, N. J.
, Atzori
, M.
, and Müller
, H.
, 2020
, “A Large Calibrated Database of Hand Movements and Grasps Kinematics
,” Sci. Data
, 7
(1
), pp. 1
–10
. 30.
Fischer
, G.
, Jermann
, D.
, List
, R.
, Reissner
, L.
, and Calcagni
, M.
, 2020
, “Development and Application of a Motion Analysis Protocol for the Kinematic Evaluation of Basic and Functional Hand and Finger Movements Using Motion Capture in a Clinical Setting—A Repeatability Study
,” Appl. Sci.
, 10
(18
), p. 6436
. 31.
Coupier
, J.
, Hamoudi
, S.
, Telese-Izzi
, S.
, Feipel
, V.
, Rooze
, M.
, and Jan
, S. V. S.
, 2016
, “A Novel Method for In-Vivo Evaluation of Finger Kinematics Including Definition of Healthy Motion Patterns
,” Clin. Biomech.
, 31
, pp. 47
–58
. 32.
Garrido-Jurado
, S.
, Muñoz-Salinas
, R.
, Madrid-Cuevas
, F. J.
, and Marín-Jiménez
, M. J.
, 2014
, “Automatic Generation and Detection of Highly Reliable Fiducial Markers Under Occlusion
,” Pattern Recognit.
, 47
(6
), pp. 2280
–2292
. 33.
Elangovan
, N.
, Dwivedi
, A.
, Gerez
, L.
, Chang
, C. M.
, and Liarokapis
, M.
, 2019
, “Employing imu and Aruco Marker Based Tracking to Decode the Contact Forces Exerted by Adaptive Hands
,” Proceedings of IEEE-RAS 19th International Conference on Humanoid Robots (Humanoids)
, Toronto, Canada
, Oct. 15–17
, pp. 525
–530
.34.
Mueller
, F.
, Bernard
, F.
, Sotnychenko
, O.
, Mehta
, D.
, Sridhar
, S.
, Casas
, D.
, and Theobalt
, C.
, 2018
, “Ganerated Hands for Real-Time 3d Hand Tracking From Monocular rgb
,” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition
, Salt Lake City, UT
, June 18–23
, pp. 49
–59
.35.
Yuan
, S.
, Ye
, Q.
, Stenger
, B.
, Jain
, S.
, and Kim
, T. K.
, 2017
, “Bighand2. 2m Benchmark: Hand Pose Dataset and State of the art Analysis
,” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.
, Honolulu, HI
, July 21–26
, pp. 4866
–4874
.36.
Garcia-Hernando
, G.
, Yuan
, S.
, Baek
, S.
, and Kim
, T. K.
, 2018
, “First-Person Hand Action Benchmark With rgb-d Videos and 3d Hand Pose Annotations
,” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition
, Salt Lake City, UT
, June 18–23
, pp. 409
–419
.37.
Huysmans
, T.
, and Molenbroek
, J. F. M.
, “DINED/Anthropometry in Design
,” https://dined.io.tudelft.nl/en, Accessed August 10, 2021.38.
Oberkampf
, D.
, DeMenthon
, D. F.
, and Davis
, L. S.
, 1996
, “Iterative Pose Estimation Using Coplanar Feature Points
,” Comput. Vis. Image Understand.
, 63
(3
), pp. 495
–511
. 39.
Cooney
, W. P.
, III, and Chao
, E. Y.
, 1977
, “Biomechanical Analysis of Static Forces in the Thumb During Hand Function
,” J. Bone Jt. Surg.
American Volume, 59
(1
), pp. 27
–36
. 40.
Chao
, E. Y. S.
, An
, K.-N.
, Cooney
, W. P.
, and Linscheid
, R. L.
, 1989
, Biomechanics of the Hand
, Y. S.
Cappo
, ed., World Scientific
, Singapore
, pp. 73
–96
.41.
Wu
, G.
, Van der Helm
, F. C.
, Veeger
, H. D.
, Makhsous
, M.
, Van Roy
, P.
, Anglin
, C.
, Nagels
, J.
, et al, 2005
, “ISB Recommendation on Definitions of Joint Coordinate Systems of Various Joints for the Reporting of Human Joint Motion—Part II: Shoulder, Elbow, Wrist and Hand
,” J. Biomech.
, 38
(5
), pp. 981
–992
. 42.
McGinley
, J. L.
, Baker
, R.
, Wolfe
, R.
, and Morris
, M. E.
, 2009
, “The Reliability of Three-Dimensional Kinematic Gait Measurements: A Systematic Review
,” Gait Posture
, 29
(3
), pp. 360
–369
. 43.
Cheng
, P. L.
, and Pearcy
, M.
, 1999
, “A Three-Dimensional Definition for the Flexion/Extension and Abduction/Adduction Angles
,” Med. Biol. Eng. Comput.
, 37
(4
), pp. 440
–444
. 44.
Bain
, G. I.
, Polites
, N.
, Higgs
, B. G.
, Heptinstall
, R. J.
, and McGrath
, A. M.
, 2015
, “The Functional Range of Motion of the Finger Joints
,” J. Hand Surg.
(European Volume), 40
(4
), pp. 406
–411
. 45.
Mueller
, F.
, Mehta
, D.
, Sotnychenko
, O.
, Sridhar
, S.
, Casas
, D.
, and Theobalt
, C.
, 2017
, “Real-Time Hand Tracking Under Occlusion From an Egocentric rgb-d Sensor
,” Proceedings of the IEEE International Conference on Computer Vision
, Venice, Italy
, Oct. 22–29
, pp. 1154
–1163
.Copyright © 2022 by ASME
You do not currently have access to this content.
