Presentation 2021-05-13
Parallelism optimization method for analytical processing on hybrid cloud.
Kaori Murase, Shinichi, Satoshi Kaneko, Kouichi Murayama,
PDF Download Page PDF download Page Link
Abstract(in Japanese) (See Japanese page)
Abstract(in English) A hybrid data analytics platform that consists of data lake on on-premise and servers for analytical processing on public cloud has the advantages of both detailed data management and flexible performance adjustment. However, for some cases cost performance declines due to design policy differences between on-premise and public cloud. For instance, cases where degree of parallelism for data analytics is set higher than necessary since it is difficult to change amount of processing resources of on-premise immediately. In this paper, we propose an optimization method for determining degree of parallelism based on allocatable on-premise resources of the hybrid data analytics platform. We prospect that it can predict appropriate degree of parallelism, and achieve an improved cost performance.
Keyword(in Japanese) (See Japanese page)
Keyword(in English) Data analytics platform / Hybrid cloud storage / Parallelism / ETL / Optimization
Paper # ICM2021-3
Date of Issue 2021-05-06 (ICM)

Conference Information
Committee ICM / IPSJ-CSEC / IPSJ-IOT
Conference Date 2021/5/13(2days)
Place (in Japanese) (See Japanese page)
Place (in English) Online
Topics (in Japanese) (See Japanese page)
Topics (in English)
Chair Kazuhiko Kinoshita(Tokushima Univ.)
Vice Chair Haruo Ooishi(NTT) / Eiji Takahashi(NEC)
Secretary Haruo Ooishi(BOSCO Technologies) / Eiji Takahashi(Fujitsu Lab.)
Assistant Yoshifumi Kato(NTT)

Paper Information
Registration To Technical Committee on Information and Communication Management / Special Interest Group on Computer Security / Special Interest Group on Internet and Operation Technology
Language JPN
Title (in Japanese) (See Japanese page)
Sub Title (in Japanese) (See Japanese page)
Title (in English) Parallelism optimization method for analytical processing on hybrid cloud.
Sub Title (in English)
Keyword(1) Data analytics platform
Keyword(2) Hybrid cloud storage
Keyword(3) Parallelism
Keyword(4) ETL
Keyword(5) Optimization
1st Author's Name Kaori Murase
1st Author's Affiliation Hitachi, Ltd. Research & Development Group(Hitachi, Ltd.)
2nd Author's Name Shinichi
2nd Author's Affiliation Hitachi, Ltd. Research & Development Group(Hitachi, Ltd.)
3rd Author's Name Satoshi Kaneko
3rd Author's Affiliation Hitachi, Ltd. Research & Development Group(Hitachi, Ltd.)
4th Author's Name Kouichi Murayama
4th Author's Affiliation Hitachi, Ltd. Research & Development Group(Hitachi, Ltd.)
Date 2021-05-13
Paper # ICM2021-3
Volume (vol) vol.121
Number (no) ICM-13
Page pp.pp.13-18(ICM),
#Pages 6
Date of Issue 2021-05-06 (ICM)