| Presentation | 2010-11-18 Understanding the Characteristics of Network Workload for MapReduce Tatsuya MORI, Tatsuaki KIMURA, Yasuhiro IKEDA, Noriaki KAMIYAMA, Ryoichi KAWAHARA, |
|---|---|
| PDF Download Page |  PDF download Page Link |
| Abstract(in Japanese) | (See Japanese page) |
| Abstract(in English) | This work studies the workloads of a distributed computing system that is used to execute MapReduce programs for processing large-scale data. Especially, we focus our attention on the network workload. A Hadoop cluster consisting of 12 nodes is used for our analysis. MapReduce job traces are collected on a master server and slave servers. To study the detailed characteristics of network load, we also collect packet header traces on the slave servers. First, through a case study analysis, we reveal the correlation between MapReduce tasks and workloads on the underlying network. Next, we show the parameter configuration of MapReduce could affect the properties of TCP flows used for the communication and data copying among servers. Finally, we discuss the implications on network measurement schemes for MapReduce-like systems. |
| Keyword(in Japanese) | (See Japanese page) |
| Keyword(in English) | MapReduce / Cloud computing / measurement / workload characterization |
| Paper # | CQ2010-49 |
| Date of Issue |
| Conference Information | |
| Committee | CQ |
|---|---|
| Conference Date | 2010/11/11(1days) |
| Place (in Japanese) | (See Japanese page) |
| Place (in English) | |
| Topics (in Japanese) | (See Japanese page) |
| Topics (in English) | |
| Chair | |
| Vice Chair | |
| Secretary | |
| Assistant | |
| Paper Information | |
| Registration To | Communication Quality (CQ) |
|---|---|
| Language | ENG |
| Title (in Japanese) | (See Japanese page) |
| Sub Title (in Japanese) | (See Japanese page) |
| Title (in English) | Understanding the Characteristics of Network Workload for MapReduce |
| Sub Title (in English) | |
| Keyword(1) | MapReduce |
| Keyword(2) | Cloud computing |
| Keyword(3) | measurement |
| Keyword(4) | workload characterization |
| 1st Author's Name | Tatsuya MORI |
| 1st Author's Affiliation | NTT Service Integration Laboratories() |
| 2nd Author's Name | Tatsuaki KIMURA |
| 2nd Author's Affiliation | NTT Service Integration Laboratories |
| 3rd Author's Name | Yasuhiro IKEDA |
| 3rd Author's Affiliation | NTT Service Integration Laboratories |
| 4th Author's Name | Noriaki KAMIYAMA |
| 4th Author's Affiliation | NTT Service Integration Laboratories |
| 5th Author's Name | Ryoichi KAWAHARA |
| 5th Author's Affiliation | NTT Service Integration Laboratories |
| Date | 2010-11-18 |
| Paper # | CQ2010-49 |
| Volume (vol) | vol.110 |
| Number (no) | 287 |
| Page | pp.pp.- |
| #Pages | 6 |
| Date of Issue | |