| Presentation | 2021-05-13 Parallelism optimization method for analytical processing on hybrid cloud. Kaori Murase, Shinichi, Satoshi Kaneko, Kouichi Murayama, |
|---|---|
| PDF Download Page |  PDF download Page Link |
| Abstract(in Japanese) | (See Japanese page) |
| Abstract(in English) | A hybrid data analytics platform that consists of data lake on on-premise and servers for analytical processing on public cloud has the advantages of both detailed data management and flexible performance adjustment. However, for some cases cost performance declines due to design policy differences between on-premise and public cloud. For instance, cases where degree of parallelism for data analytics is set higher than necessary since it is difficult to change amount of processing resources of on-premise immediately. In this paper, we propose an optimization method for determining degree of parallelism based on allocatable on-premise resources of the hybrid data analytics platform. We prospect that it can predict appropriate degree of parallelism, and achieve an improved cost performance. |
| Keyword(in Japanese) | (See Japanese page) |
| Keyword(in English) | Data analytics platform / Hybrid cloud storage / Parallelism / ETL / Optimization |
| Paper # | ICM2021-3 |
| Date of Issue | 2021-05-06 (ICM) |
| Conference Information | |
| Committee | ICM / IPSJ-CSEC / IPSJ-IOT |
|---|---|
| Conference Date | 2021/5/13(2days) |
| Place (in Japanese) | (See Japanese page) |
| Place (in English) | Online |
| Topics (in Japanese) | (See Japanese page) |
| Topics (in English) | |
| Chair | Kazuhiko Kinoshita(Tokushima Univ.) |
| Vice Chair | Haruo Ooishi(NTT) / Eiji Takahashi(NEC) |
| Secretary | Haruo Ooishi(BOSCO Technologies) / Eiji Takahashi(Fujitsu Lab.) |
| Assistant | Yoshifumi Kato(NTT) |
| Paper Information | |
| Registration To | Technical Committee on Information and Communication Management / Special Interest Group on Computer Security / Special Interest Group on Internet and Operation Technology |
|---|---|
| Language | JPN |
| Title (in Japanese) | (See Japanese page) |
| Sub Title (in Japanese) | (See Japanese page) |
| Title (in English) | Parallelism optimization method for analytical processing on hybrid cloud. |
| Sub Title (in English) | |
| Keyword(1) | Data analytics platform |
| Keyword(2) | Hybrid cloud storage |
| Keyword(3) | Parallelism |
| Keyword(4) | ETL |
| Keyword(5) | Optimization |
| 1st Author's Name | Kaori Murase |
| 1st Author's Affiliation | Hitachi, Ltd. Research & Development Group(Hitachi, Ltd.) |
| 2nd Author's Name | Shinichi |
| 2nd Author's Affiliation | Hitachi, Ltd. Research & Development Group(Hitachi, Ltd.) |
| 3rd Author's Name | Satoshi Kaneko |
| 3rd Author's Affiliation | Hitachi, Ltd. Research & Development Group(Hitachi, Ltd.) |
| 4th Author's Name | Kouichi Murayama |
| 4th Author's Affiliation | Hitachi, Ltd. Research & Development Group(Hitachi, Ltd.) |
| Date | 2021-05-13 |
| Paper # | ICM2021-3 |
| Volume (vol) | vol.121 |
| Number (no) | ICM-13 |
| Page | pp.pp.13-18(ICM), |
| #Pages | 6 |
| Date of Issue | 2021-05-06 (ICM) |