Presentation 2021-06-28
Training Neural ODE by Symplectic Integrator
Takashi Matsubara, Yuto Miyatake, Takaharu Yaguchi,
PDF Download Page PDF download Page Link
Abstract(in Japanese) (See Japanese page)
Abstract(in English) A differential equation model using neural networks, neural ODE, enables use to model a continuous-time dynamics and probabilistic model with high accuracy. However, the neural ODE uses the same neural network repeatedly, the training using the backpropagation algorithm consumes large memory. Instead of the backpropagation algorithm, the adjoint method is commonly used, which obtains the gradient using the numerical integration. The adjoint method needs a small step size and much computational cost to suppress the numerical errors. In this study, we combine the checkpointing scheme and symplectic integrator for the adjoint method. It suppresses the memory consumption and functions faster.
Keyword(in Japanese) (See Japanese page)
Keyword(in English) Neural ODE / ordinary differential equation / adjoint method / symplectic integrator
Paper # NC2021-2,IBISML2021-2
Date of Issue 2021-06-21 (NC, IBISML)

Conference Information
Committee NC / IBISML / IPSJ-BIO / IPSJ-MPS
Conference Date 2021/6/28(3days)
Place (in Japanese) (See Japanese page)
Place (in English) Online
Topics (in Japanese) (See Japanese page)
Topics (in English)
Chair Rieko Osu(Waseda Univ.) / Ichiro Takeuchi(Nagoya Inst. of Tech.) / 倉田 博之(九工大) / 関嶋 政和(東工大)
Vice Chair Hiroshi Yamakawa(Univ of Tokyo) / Masashi Sugiyama(Univ. of Tokyo)
Secretary Hiroshi Yamakawa(ATR) / Masashi Sugiyama(NICT) / (Univ. of Tokyo) / (AIST)
Assistant Nobuhiko Wagatsuma(Toho Univ.) / Tomoki Kurikawa(KMU) / Tomoharu Iwata(NTT) / Atsuyoshi Nakamura(Hokkaido Univ.)

Paper Information
Registration To Technical Committee on Neurocomputing / Technical Committee on Infomation-Based Induction Sciences and Machine Learning / Special Interest Group on Bioinformatics and Genomics / Special Interest Group on Mathematical Modeling and Problem Solving
Language JPN
Title (in Japanese) (See Japanese page)
Sub Title (in Japanese) (See Japanese page)
Title (in English) Training Neural ODE by Symplectic Integrator
Sub Title (in English)
Keyword(1) Neural ODE
Keyword(2) ordinary differential equation
Keyword(3) adjoint method
Keyword(4) symplectic integrator
1st Author's Name Takashi Matsubara
1st Author's Affiliation Osaka University(Osaka Univ.)
2nd Author's Name Yuto Miyatake
2nd Author's Affiliation Osaka University(Osaka Univ.)
3rd Author's Name Takaharu Yaguchi
3rd Author's Affiliation Kobe University(Kobe Univ.)
Date 2021-06-28
Paper # NC2021-2,IBISML2021-2
Volume (vol) vol.121
Number (no) NC-79,IBISML-80
Page pp.pp.9-14(NC), pp.9-14(IBISML),
#Pages 6
Date of Issue 2021-06-21 (NC, IBISML)