| Presentation | 1994/3/25 EFFECTS OF ACTIVATION FUNCTIONS IN MULTILAYER NEURAL NETWORK FOR NOISY PATTERN CLASSIFICATION Kazuyuki Hara, Kenji Nakayama, |
|---|---|
| PDF Download Page | PDF download Page Link |
| Abstract(in Japanese) | (See Japanese page) |
| Abstract(in English) | This paper discusses properties of activation functions in multilayer neural network applied to multi-frequency classification.A rule of thumb for selecting activation functions or their combination is proposed.The sigmoid,Gaussian and sinusoidal functions are employed due to their unique space division properties.Properties of each function and their combinations are discussed based on the internal representation, that is the distributions of the hidden unit inputs and outputs, classification rates with and without noise and the connection weight analysis.The sigmoid function is not effective for a single hidden unit.On the contrary,the other functions can provide good performance.When several hidden units are employed,the sigmqid function becomes useful.However,the convergence speed is still slower than the others.The Gaussian function is sensitive to the additive noise,while the others are rather insensitive.When noise is not included,the Gaussian function is most useful for the convergence rate and the classification accuracy.On the other hand, the additive noise is included,the sigmoid and sinusoidal functions become more effective.These properties are not straight in the combinations.However,their property still remain,and it is possible to select the optimum activation function.This selection also depends on the patterns to be classified. |
| Keyword(in Japanese) | (See Japanese page) |
| Keyword(in English) | Multilayer Neural Network / Pattern classification / Multi- frequency Signal / Activation function / Back propagation |
| Paper # | NC93-114 |
| Date of Issue |
| Conference Information | |
| Committee | NC |
|---|---|
| Conference Date | 1994/3/25(1days) |
| Place (in Japanese) | (See Japanese page) |
| Place (in English) | |
| Topics (in Japanese) | (See Japanese page) |
| Topics (in English) | |
| Chair | |
| Vice Chair | |
| Secretary | |
| Assistant | |
| Paper Information | |
| Registration To | Neurocomputing (NC) |
|---|---|
| Language | ENG |
| Title (in Japanese) | (See Japanese page) |
| Sub Title (in Japanese) | (See Japanese page) |
| Title (in English) | EFFECTS OF ACTIVATION FUNCTIONS IN MULTILAYER NEURAL NETWORK FOR NOISY PATTERN CLASSIFICATION |
| Sub Title (in English) | |
| Keyword(1) | Multilayer Neural Network |
| Keyword(2) | Pattern classification |
| Keyword(3) | Multi- frequency Signal |
| Keyword(4) | Activation function |
| Keyword(5) | Back propagation |
| 1st Author's Name | Kazuyuki Hara |
| 1st Author's Affiliation | Groduate School of Natural Science and Technology,Kanazawa University() |
| 2nd Author's Name | Kenji Nakayama |
| 2nd Author's Affiliation | Department of Electrical and Computer Engineering,Faculty of Technology,Kanazawa University |
| Date | 1994/3/25 |
| Paper # | NC93-114 |
| Volume (vol) | vol.93 |
| Number (no) | 537 |
| Page | pp.pp.- |
| #Pages | 8 |
| Date of Issue | |