| Presentation | 2020-01-09 Real Log Canonical Threshold of Three Layered Neural Network with Swish Activation Function Raiki Tanaka, Sumio Watanabe, |
|---|---|
| PDF Download Page |  PDF download Page Link |
| Abstract(in Japanese) | (See Japanese page) |
| Abstract(in English) | In neural network learning, it is known that selection of activation function effects generalization performance. Although a ReLU function is often employed in many applications, a new Swish function was found by reinforcement learning by Google Brain team. It is experimentally shown that neural newtorks using Swith function have better performance than other activation functions in image recognition and machine translation. However, the theoretical property of Swith function has not yet been studied. In this paper, we derive the real log canonical threshold of Swith function using algebraic geomeric method, and clafiy the generalization error and the free energy in Bayesian estimation. |
| Keyword(in Japanese) | (See Japanese page) |
| Keyword(in English) | Bayesian Learning / Neural Network / Real Log Canonical Threshold / Bayesian Generalization Error / Free Energy |
| Paper # | IBISML2019-19 |
| Date of Issue | 2020-01-02 (IBISML) |
| Conference Information | |
| Committee | IBISML |
|---|---|
| Conference Date | 2020/1/9(2days) |
| Place (in Japanese) | (See Japanese page) |
| Place (in English) | ISM |
| Topics (in Japanese) | (See Japanese page) |
| Topics (in English) | Machine learning, etc. |
| Chair | Hisashi Kashima(Kyoto Univ.) |
| Vice Chair | Masashi Sugiyama(Univ. of Tokyo) / Koji Tsuda(Univ. of Tokyo) |
| Secretary | Masashi Sugiyama(Nagoya Inst. of Tech.) / Koji Tsuda(AIST) |
| Assistant | Tomoharu Iwata(NTT) / Shigeyuki Oba(Kyoto Univ.) |
| Paper Information | |
| Registration To | Technical Committee on Infomation-Based Induction Sciences and Machine Learning |
|---|---|
| Language | JPN |
| Title (in Japanese) | (See Japanese page) |
| Sub Title (in Japanese) | (See Japanese page) |
| Title (in English) | Real Log Canonical Threshold of Three Layered Neural Network with Swish Activation Function |
| Sub Title (in English) | |
| Keyword(1) | Bayesian Learning |
| Keyword(2) | Neural Network |
| Keyword(3) | Real Log Canonical Threshold |
| Keyword(4) | Bayesian Generalization Error |
| Keyword(5) | Free Energy |
| 1st Author's Name | Raiki Tanaka |
| 1st Author's Affiliation | Tokyo Institute of Technology(Tokyo Tech) |
| 2nd Author's Name | Sumio Watanabe |
| 2nd Author's Affiliation | Tokyo Institute of Technology(Tokyo Tech) |
| Date | 2020-01-09 |
| Paper # | IBISML2019-19 |
| Volume (vol) | vol.119 |
| Number (no) | IBISML-360 |
| Page | pp.pp.9-15(IBISML), |
| #Pages | 7 |
| Date of Issue | 2020-01-02 (IBISML) |