Existing techniques for motion imitation often suffer a certain level of latency due to their computational overhead or a large set of correspondence samples to search. To achieve real-time imitation with small latency, we present a framework in this paper to reconstruct motion on humanoids based on sparsely sampled correspondence. The imitation problem is formulated as finding the projection of a point from the configuration space of a human's poses into the configuration space of a humanoid. An optimal projection is defined as the one that minimizes a back-projected deviation among a group of candidates, which can be determined in a very efficient way. Benefited from this formulation, effective projections can be obtained by using sparsely sampled correspondence, whose generation scheme is also introduced in this paper. Our method is evaluated by applying the human's motion captured by an RGB-depth (RGB-D) sensor to a humanoid in real time. Continuous motion can be realized and used in the example application of teleoperation.
Issue Section:
Research Papers
Topics:
Haptics,
Machinery,
Sensors,
Virtual reality,
Kinematics,
Robots,
Humanoid robots,
Dimensions,
Optimization,
Space,
Statistics
References
1.
Suleiman
, W.
, Yoshida
, E.
, Kanehiro
, F.
, Laumond
, J.-P.
, and Monin
, A.
, 2008
, “On Human Motion Imitation by Humanoid Robot
,” IEEE International Conference on Robotics and Automation
(ICRA
), Pasadena, CA, May 19–23, pp. 2697
–2704
.2.
Chalodhorn
, R.
, Grimes
, D. B.
, Grochow
, K.
, and Rao
, R. P.
, 2007, “Learning to Walk Through Imitation
,” 20th International Joint Conference on Artifical Intelligence (IJCAI
), Hyderabad, India, Jan. 6–12, Vol. 7
, pp. 2084
–2090
.3.
Nakaoka
, S.
, Nakazawa
, A.
, Kanehiro
, F.
, Kaneko
, K.
, Morisawa
, M.
, Hirukawa
, H.
, and Ikeuchi
, K.
, 2007
, “Learning From Observation Paradigm: Leg Task Models for Enabling a Biped Humanoid Robot to Imitate Human Dances
,” Int. J. Rob. Res.
, 26
(8
), pp. 829
–844
.4.
Ude
, A.
, Atkeson
, C. G.
, and Riley
, M.
, 2004
, “Programming Full-Body Movements for Humanoid Robots by Observation
,” Rob. Auton. Syst.
, 47
(2–3
), pp. 93
–108
.5.
Kim
, S.
, Kim
, C.
, You
, B.
, and Oh
, S.
, 2009
, “Stable Whole-Body Motion Generation for Humanoid Robots to Imitate Human Motions
,” IEEE/RSJ International Conference on Intelligent Robots and Systems
(IROS
), St. Louis, MO, Oct. 10–15, pp. 2518
–2524
.6.
Safonova
, A.
, Pollard
, N. S.
, and Hodgins
, J. K.
, 2003
, “Optimizing Human Motion for the Control of a Humanoid Robot
,” 2nd International Symposium on Adaptive Motion of Animals and machines (AMAM
), Kyoto, Japan, Mar. 4–8.7.
Ott
, C.
, Lee
, D.
, and Nakamura
, Y.
, 2008
, “Motion Capture Based Human Motion Recognition and Imitation by Direct Marker Control
,” Eighth IEEE-RAS
International Conference on Humanoid Robots, Daejeon, South Korea, Dec. 1–3, pp. 399
–405
.8.
Dariush
, B.
, Gienger
, M.
, Arumbakkam
, A.
, Zhu
, Y.
, Jian
, B.
, Fujimura
, K.
, and Goerick
, C.
, 2009
, “Online Transfer of Human Motion to Humanoids
,” Int. J. Humanoid Rob.
, 6
(2
), pp. 265
–289
.9.
Do
, M.
, Azad
, P.
, Asfour
, T.
, and Dillmann
, R.
, 2008
, “Imitation of Human Motion on a Humanoid Robot Using Non-Linear Optimization
,” Eighth IEEE-RAS
International Conference on Humanoid Robots
, Daejeon, South Korea, Dec. 1–3, pp. 545
–552
.10.
Yamane
, K.
, Anderson
, S. O.
, and Hodgins
, J. K.
, 2010
, “Controlling Humanoid Robots With Human Motion Data: Experimental Validation
,” Tenth IEEE-RAS
International Conference on Humanoid Robots
, Nashville, TN, Dec. 6–8, pp. 504
–510
.11.
Koenemann
, J.
, and Bennewitz
, M.
, 2012
, “Whole-Body Imitation of Human Motions With a Nao Humanoid
,” Seventh ACM/IEEE International Conference on Human-Robot Interaction
(HRI
), Boston, MA, Mar. 5–8, pp. 425
–425
.12.
Koenemann
, J.
, Burget
, F.
, and Bennewitz
, M.
, 2014
, “Real-Time Imitation of Human Whole-Body Motions by Humanoids
,” IEEE International Conference on Robotics and Automation
(ICRA
), Hong Kong, China, May 31–June 7, pp. 2806
–2812
.13.
Morris
, A. S.
, and Mansor
, A.
, 1997
, “Finding the Inverse Kinematics of Manipulator Arm Using Artificial Neural Network With Lookup Table
,” Robotica
, 26
(6
), pp. 617
–625
.14.
Aleotti
, J.
, Skoglund
, A.
, and Duckett
, T.
, 2004
, “Position Teaching of a Robot Arm by Demonstration With a Wearable Input Device
,” International Conference on Intelligent Manipulation and Grasping
(IMG
), Genova, Italy, July 1–2.15.
Neto
, P.
, Pires
, J. N.
, and Moreira
, A. P.
, 2009
, “Accelerometer-Based Control of an Industrial Robotic Arm
,” 18th IEEE International Symposium on Robot and Human Interactive Communication
(RO-MAN
), Toyama, Japan, Sept. 27–Oct. 2, pp. 1192
–1197
.16.
Neto
, P.
, Pires
, J. N.
, and Moreira
, A. P.
, 2010
, “High-Level Programming and Control for Industrial Robotics: Using a Hand-Held Accelerometer-Based Input Device for Gesture and Posture Recognition
,” Ind. Rob.: Int. J.
, 37
(2
), pp. 137
–147
.17.
Stanton
, C.
, Bogdanovych
, A.
, and Ratanasena
, E.
, 2012
, “Teleoperation of a Humanoid Robot Using Full-Body Motion Capture, Example Movements, and Machine Learning
,” Australasian Conference on Robotics and Automation
(ACRA
), Wellington, New Zealand, Dec. 3–5, pp. 260–269.18.
Van der Smagt
, P.
, and Schulten
, K.
, 1993, “Control of Pneumatic Robot Arm Dynamics by a Neural Network
,” World Congress on Neural Networks
, Portland, OR, July 11–15, Vol. 3
, pp. 180
–183
.19.
Jung
, S.
, and Hsia
, T.
, 1996
, “Neural Network Reference Compensation Technique for Position Control of Robot Manipulators
,” IEEE International Conference on Neural Networks
(ICNN
), Washington, DC, June 3–6, Vol. 3
, pp. 1765
–1770
.20.
Larsen
, J. C.
, and Ferrier
, N. J.
, 2004
, “A Case Study in Vision Based Neural Network Training for Control of a Planar, Large Deflection, Flexible Robot Manipulator
,” IEEE/RSJ International Conference on Intelligent Robots and Systems
(IROS
), Sendai, Japan, Sept. 28–Oct. 2, Vol. 3
, pp. 2924
–2929
.21.
Wang
, D.
, and Bai
, Y.
, 2005
, “Improving Position Accuracy of Robot Manipulators Using Neural Networks
,” IEEE Instrumentation and Measurement Technology Conference
(IMTC
), Ottawa, ON, Canada, May 16–19, Vol. 2
, pp. 1524
–1526
.22.
Comaniciu
, D.
, and Meer
, P.
, 2002
, “Mean Shift: A Robust Approach Toward Feature Space Analysis
,” IEEE Trans. Pattern Anal. Mach. Intell.
, 24
(5
), pp. 603
–619
.23.
Huang
, G.-B.
, Wang
, D. H.
, and Lan
, Y.
, 2011
, “Extreme Learning Machines: A Survey
,” Int. J. Mach. Learn. Cybern.
, 2
(2
), pp. 107
–122
.24.
Huang
, G.-B.
, Zhou
, H.
, Ding
, X.
, and Zhang
, R.
, 2012
, “Extreme Learning Machine for Regression and Multiclass Classification
,” IEEE Trans. Syst., Man, Cybern., Part B: Cybern.
, 26
(2
), pp. 513
–529
. 25.
Deng
, W.
, Zheng
, Q.
, and Chen
, L.
, 2009
, “Regularized Extreme Learning Machine
,” IEEE Symposium on Computational Intelligence and Data Mining
(CIDM
), Nashville, TN, Mar. 30–Apr. 2, pp. 389
–395
.26.
LaViola
, J. J.
, 2003
, “Double Exponential Smoothing: An Alternative to Kalman Filter-Based Predictive Tracking
,” ACM
Workshop on Virtual Environments
, Zurich, Switzerland, May 22–23, pp. 199
–206
.27.
Lapeyre, M., Rouanet, P., Grizou, J., Nguyen, S., Depraetre, F., Le Falher, A., and Oudeyer, P.-Y., 2014, “
Poppy Project: Open-Source Fabrication of 3D Printed Humanoid Robot for Science, Education and Art
,” Digital Intelligence (DI2014
), Nantes, France, Sept. 17–19.28.
Yeung
, K.-Y.
, Kwok
, T.-H.
, and Wang
, C. C.
, 2013
, “Improved Skeleton Tracking by Duplex Kinects: A Practical Approach for Real-Time Applications
,” ASME J. Comput. Inf. Sci. Eng.
, 13
(4
), p. 041007
.29.
Zheng
, Y.
, Chan
, K. C.
, and Wang
, C. C. L.
, 2014
, “Pedalvatar: An IMU-Based Real-Time Body Motion Capture System Using Foot Rooted Kinematic Model
,” IEEE/RSJ International Conference on Intelligent Robots and Systems
(IROS
), Chicago, IL, Sept. 14–18, pp. 4130
–4135
.Copyright © 2017 by ASME
You do not currently have access to this content.
