Presentation 2023-02-22
A Study of Estimating 3D Skeleton of a Human Full-Body Using Images Acquired by Omnidirectional Cameras Attached to the Human Body and Deep Learning
Yuta Arai, Jun Ohya, Hiroyuki Ogata, Seo Chanjin,
PDF Download Page PDF download Page Link
Abstract(in Japanese) (See Japanese page)
Abstract(in English) In recent years, motion capture has been used for training in various sporting events. However, in sporting events involving extensive movement, there are problems such as the field of view of a fixed camera not being able to fit. To solve such problems, this article proposes a new method for estimating the 3D skeleton by processing 360-degree images acquired by an omnidirectional camera attached to the body. In the proposed method, we do a learning of the CNN (Convolutional Neural Network) using the results of annotating the joint points of the human body in the images and the images from the omnidirectional camera. The 3D coordinates of each joint point are output by inputting the omnidirectional camera images to the trained CNN model. We conducted 3D skeleton estimation experiments by acquiring data in both simulated and real environments. As a result of the experiments, a confidence rate of more than 90% was achieved within an error margin of 3 cm, we have indicated that the method is effective.
Keyword(in Japanese) (See Japanese page)
Keyword(in English) Motion Capture / Skeleton Estimation / Deep Learning / CNN / Omnidirectional Camera / Unity / Azure Kinect / First Person View
Paper # ITS2022-62,IE2022-79
Date of Issue 2023-02-14 (ITS, IE)

Conference Information
Committee IE / ITS / ITE-MMS / ITE-ME / ITE-AIT
Conference Date 2023/2/21(2days)
Place (in Japanese) (See Japanese page)
Place (in English) Hokkaido Univ.
Topics (in Japanese) (See Japanese page)
Topics (in English) Image Processing, etc.
Chair Kazuya Kodama(NII) / Masahiro Fujii(Utsunomiya Univ.) / Kenji Machida(NHK) / Hiroyuki Arai(Nippon Inst. of Tech.) / Hisaki Nate(Tokyo Polytechnic Univ.)
Vice Chair Hiroyuki Bandoh(NTT) / Toshihiko Yamazaki(Univ. of Tokyo) / Kohei Ohno(Meiji Univ.) / Naohisa Hashimoto(AIST) / / Shogo Muramatsu(Niigata Univ)
Secretary Hiroyuki Bandoh(KDDI Research) / Toshihiko Yamazaki(Nagoya Inst. of Tech.) / Kohei Ohno(Toyama Prefectural Univ.) / Naohisa Hashimoto(NIT, Tsuruoka College) / (Fukuoka Univ.) / Shogo Muramatsu(NHK) / (Hokkaido Univ.)
Assistant Shunsuke Iwamura(NHK) / Shinobu Kudo(NTT) / Taishi Swabe(NAIST) / Keiji Jimi(Gunma Univ.)

Paper Information
Registration To Technical Committee on Image Engineering / Technical Committee on Intelligent Transport Systems Technology / Technical Group on Multi-media Storage / Technical Group on Media Engineering / Technical Group on Artistic Image Technology
Language JPN
Title (in Japanese) (See Japanese page)
Sub Title (in Japanese) (See Japanese page)
Title (in English) A Study of Estimating 3D Skeleton of a Human Full-Body Using Images Acquired by Omnidirectional Cameras Attached to the Human Body and Deep Learning
Sub Title (in English)
Keyword(1) Motion Capture
Keyword(2) Skeleton Estimation
Keyword(3) Deep Learning
Keyword(4) CNN
Keyword(5) Omnidirectional Camera
Keyword(6) Unity
Keyword(7) Azure Kinect
Keyword(8) First Person View
1st Author's Name Yuta Arai
1st Author's Affiliation Waseda University(Waseda Univ.)
2nd Author's Name Jun Ohya
2nd Author's Affiliation Waseda University(Waseda Univ.)
3rd Author's Name Hiroyuki Ogata
3rd Author's Affiliation Seikei University(Seikei Univ.)
4th Author's Name Seo Chanjin
4th Author's Affiliation Waseda University(Waseda Univ.)
Date 2023-02-22
Paper # ITS2022-62,IE2022-79
Volume (vol) vol.122
Number (no) ITS-384,IE-385
Page pp.pp.112-117(ITS), pp.112-117(IE),
#Pages 6
Date of Issue 2023-02-14 (ITS, IE)