| Presentation | 2017-12-21 Consideration of Parallel Data Processing over an Apache Spark, a large-scale data distributed platform Kasumi Kato, Atsuko Takefusa, Hidemoto Nakada, Masato Oguchi, |
|---|---|
| PDF Download Page |  PDF download Page Link |
| Abstract(in Japanese) | (See Japanese page) |
| Abstract(in English) | The Spread of cameras and sensors and cloud technologies enable us to obtain life logs at ordinary homes and transmit the captured data to a cloud for life log analysis. However, the amount of processing for video data analysis in a cloud drastically increases when a very large number of homes send data to the cloud. In this research, we aim to improve the efficiency of distributed video data analysis processing by using the parallel deep learning framework Chainer and the distribution processing platform Apache Spark. In this paper, we construct a Spark cluster and investigate the performance of parallel data processing using Spark varying parameter settings. |
| Keyword(in Japanese) | (See Japanese page) |
| Keyword(in English) | distributed processing / parallel processing / Spark / Chainer |
| Paper # | MoNA2017-40 |
| Date of Issue | 2017-12-14 (MoNA) |
| Conference Information | |
| Committee | MoNA |
|---|---|
| Conference Date | 2017/12/21(1days) |
| Place (in Japanese) | (See Japanese page) |
| Place (in English) | Ochanomizu University |
| Topics (in Japanese) | (See Japanese page) |
| Topics (in English) | Cloud computing, Big data, Wireless network, etc. |
| Chair | Ryoichi Shinkuma(Kyoto Univ.) |
| Vice Chair | Shigeaki Tagashira(Kansai Univ.) / Gen Kitagata(Tohoku Univ.) |
| Secretary | Shigeaki Tagashira(Kyushu Univ.) / Gen Kitagata(NTT) |
| Assistant | Takayuki Nishio(Kyoto Univ.) / Takato Saito(NTT) |
| Paper Information | |
| Registration To | Technical Committee on Mobile Network and Applications |
|---|---|
| Language | JPN |
| Title (in Japanese) | (See Japanese page) |
| Sub Title (in Japanese) | (See Japanese page) |
| Title (in English) | Consideration of Parallel Data Processing over an Apache Spark, a large-scale data distributed platform |
| Sub Title (in English) | |
| Keyword(1) | distributed processing |
| Keyword(2) | parallel processing |
| Keyword(3) | Spark |
| Keyword(4) | Chainer |
| 1st Author's Name | Kasumi Kato |
| 1st Author's Affiliation | Ochanomizu University(Ocha Univ.) |
| 2nd Author's Name | Atsuko Takefusa |
| 2nd Author's Affiliation | National Institute of Informatics(NII) |
| 3rd Author's Name | Hidemoto Nakada |
| 3rd Author's Affiliation | National Institute of Advanced Industrial Science and Technology(AIST) |
| 4th Author's Name | Masato Oguchi |
| 4th Author's Affiliation | Ochanomizu University(Ocha Univ.) |
| Date | 2017-12-21 |
| Paper # | MoNA2017-40 |
| Volume (vol) | vol.117 |
| Number (no) | MoNA-371 |
| Page | pp.pp.65-69(MoNA), |
| #Pages | 5 |
| Date of Issue | 2017-12-14 (MoNA) |