Summary
International Symposium on Nonlinear Theory and Its Applications
2022
Session Number:A3L-D
Session:
Number:A3L-D-01
Efficient Hessian Vector Products Calculation of Neural ODE Based on Second-Order Adjoint Method
Atsuhiro Hada , Satoru Iwasaki,
pp.79-82
Publication Date:12/12/2022
Online ISSN:2188-5079
Summary:
Neural ordinary differential equations (Neural ODEs) are types of the neural net architectures, and its intermediate layers are modeled as ordinary differential equations instead of discrete sequence of hidden layers. In Neural ODEs, we can not compute Hessians by conventional automatic differentiation methods since its intermediate layer possesses ODE forms. In this study, we propose methodologies to get second order derivatives of loss functions of Neural ODEs and this will allow us to analyze loss landscapes of Neural ODEs.