| Presentation | 2019-03-08 Pricing and Resource Allocation on Hadoop Data Centers Using Auction Mechanisms Yu Manabe, Jun Kawahara, Shoji Kasahara, |
|---|---|
| PDF Download Page |  PDF download Page Link |
| Abstract(in Japanese) | (See Japanese page) |
| Abstract(in English) | As storage capacity and computing performance grow due to the development of hardware, it is possible for applications to handle much more data and computation resources. Therefore, distributed processing frameworks that accelerate execution of jobs in a large sale cluster such as Hadoop attract the attention. If a large number of users share big data, a new scheduler for resource allocation is needed. In addition, in order to manage a Hadoop data center for commercial purposes, a method of pricing that increases the revenue of a data center is required. In this research, we propose a new method consisting of two components. The first component is a system to predict the execution time of a user's application. The second component is an auction mechanism such that it is optimal for users to bid their own appraised price as they are. Discrete event simulations are carried out to evaluate the proposed method. The evaluation experiment indicates that the proposed method increases the revenue of the Hadoop data center by about 125% compared with an existing method in the situation where user participation and leaving are frequent. |
| Keyword(in Japanese) | (See Japanese page) |
| Keyword(in English) | Distributed processing system / Hadoop / Data center / Scheduling / Auction mechanism |
| Paper # | ICM2018-57 |
| Date of Issue | 2019-02-28 (ICM) |
| Conference Information | |
| Committee | ICM |
|---|---|
| Conference Date | 2019/3/7(2days) |
| Place (in Japanese) | (See Japanese page) |
| Place (in English) | |
| Topics (in Japanese) | (See Japanese page) |
| Topics (in English) | |
| Chair | Kiyohito Yoshihara(KDDI Research) |
| Vice Chair | Yoichi Yamashita(NTT-Neomait) / Takumi Miyoshi(Shibaura Inst. of Tech.) |
| Secretary | Yoichi Yamashita(KDDI Research) / Takumi Miyoshi(NTT) |
| Assistant | Yunchen Zhu(Hitachi) |
| Paper Information | |
| Registration To | Technical Committee on Information and Communication Management |
|---|---|
| Language | JPN |
| Title (in Japanese) | (See Japanese page) |
| Sub Title (in Japanese) | (See Japanese page) |
| Title (in English) | Pricing and Resource Allocation on Hadoop Data Centers Using Auction Mechanisms |
| Sub Title (in English) | |
| Keyword(1) | Distributed processing system |
| Keyword(2) | Hadoop |
| Keyword(3) | Data center |
| Keyword(4) | Scheduling |
| Keyword(5) | Auction mechanism |
| 1st Author's Name | Yu Manabe |
| 1st Author's Affiliation | NARA Institute of Science and Technology(NAIST) |
| 2nd Author's Name | Jun Kawahara |
| 2nd Author's Affiliation | NARA Institute of Science and Technology(NAIST) |
| 3rd Author's Name | Shoji Kasahara |
| 3rd Author's Affiliation | NARA Institute of Science and Technology(NAIST) |
| Date | 2019-03-08 |
| Paper # | ICM2018-57 |
| Volume (vol) | vol.118 |
| Number (no) | ICM-483 |
| Page | pp.pp.43-48(ICM), |
| #Pages | 6 |
| Date of Issue | 2019-02-28 (ICM) |