| Presentation | 2002/12/6 Parallel Learning of Neural Networks on PC-Cluster Systems Using Mini-Batch Learning Schema Kohta HORISAKI, Noriyuki YAMANAKA, Ichiro TAKEUCHI, Takeshi FURUHASHI, |
|---|---|
| PDF Download Page | PDF download Page Link |
| Abstract(in Japanese) | (See Japanese page) |
| Abstract(in English) | Due to the increasing development of hardware and network technology, it has become common to have a huge data set. When processing such data by neural networks, learning takes long time. In this paper, we propose parallel learning by distributing a task into two or more CPUs. In parallel learning, mini-batch learning schema is indespensable because we must divide the entire data set into subsets and allocate each of them to the parallelized neural network. In parallel learning, the redundancy of the data is a key concept. When there are no redundancy in data, learning accuracy might decrease by the existence of the bias in each subset. On the other hand, when the data has much redundancy, the bias in each subset would be small and parallel learning might yield high efficiency. In this paper, we study a parallel learning procedure using mini-batch learning schema and investigate the relationship between the efficiency of the parallel learning and the data redundancy. It is quite common for huge data sets to have some redundancy and we expect our procedure might work in a variety of applications. |
| Keyword(in Japanese) | (See Japanese page) |
| Keyword(in English) | Neural Network / Parallel Learning / Minibatch Learning / PC-Cluster / Redundancy of Data |
| Paper # | NC2002-91 |
| Date of Issue |
| Conference Information | |
| Committee | NC |
|---|---|
| Conference Date | 2002/12/6(1days) |
| Place (in Japanese) | (See Japanese page) |
| Place (in English) | |
| Topics (in Japanese) | (See Japanese page) |
| Topics (in English) | |
| Chair | |
| Vice Chair | |
| Secretary | |
| Assistant | |
| Paper Information | |
| Registration To | Neurocomputing (NC) |
|---|---|
| Language | JPN |
| Title (in Japanese) | (See Japanese page) |
| Sub Title (in Japanese) | (See Japanese page) |
| Title (in English) | Parallel Learning of Neural Networks on PC-Cluster Systems Using Mini-Batch Learning Schema |
| Sub Title (in English) | |
| Keyword(1) | Neural Network |
| Keyword(2) | Parallel Learning |
| Keyword(3) | Minibatch Learning |
| Keyword(4) | PC-Cluster |
| Keyword(5) | Redundancy of Data |
| 1st Author's Name | Kohta HORISAKI |
| 1st Author's Affiliation | Faculty of Engineering, Mie University() |
| 2nd Author's Name | Noriyuki YAMANAKA |
| 2nd Author's Affiliation | Faculty of Engineering, Mie University |
| 3rd Author's Name | Ichiro TAKEUCHI |
| 3rd Author's Affiliation | Faculty of Engineering, Mie University |
| 4th Author's Name | Takeshi FURUHASHI |
| 4th Author's Affiliation | Faculty of Engineering, Mie University |
| Date | 2002/12/6 |
| Paper # | NC2002-91 |
| Volume (vol) | vol.102 |
| Number (no) | 508 |
| Page | pp.pp.- |
| #Pages | 4 |
| Date of Issue | |