Presentation 2010-11-18
Understanding the Characteristics of Network Workload for MapReduce
Tatsuya MORI, Tatsuaki KIMURA, Yasuhiro IKEDA, Noriaki KAMIYAMA, Ryoichi KAWAHARA,
PDF Download Page PDF download Page Link
Abstract(in Japanese) (See Japanese page)
Abstract(in English) This work studies the workloads of a distributed computing system that is used to execute MapReduce programs for processing large-scale data. Especially, we focus our attention on the network workload. A Hadoop cluster consisting of 12 nodes is used for our analysis. MapReduce job traces are collected on a master server and slave servers. To study the detailed characteristics of network load, we also collect packet header traces on the slave servers. First, through a case study analysis, we reveal the correlation between MapReduce tasks and workloads on the underlying network. Next, we show the parameter configuration of MapReduce could affect the properties of TCP flows used for the communication and data copying among servers. Finally, we discuss the implications on network measurement schemes for MapReduce-like systems.
Keyword(in Japanese) (See Japanese page)
Keyword(in English) MapReduce / Cloud computing / measurement / workload characterization
Paper # CQ2010-49
Date of Issue

Conference Information
Committee CQ
Conference Date 2010/11/11(1days)
Place (in Japanese) (See Japanese page)
Place (in English)
Topics (in Japanese) (See Japanese page)
Topics (in English)
Chair
Vice Chair
Secretary
Assistant

Paper Information
Registration To Communication Quality (CQ)
Language ENG
Title (in Japanese) (See Japanese page)
Sub Title (in Japanese) (See Japanese page)
Title (in English) Understanding the Characteristics of Network Workload for MapReduce
Sub Title (in English)
Keyword(1) MapReduce
Keyword(2) Cloud computing
Keyword(3) measurement
Keyword(4) workload characterization
1st Author's Name Tatsuya MORI
1st Author's Affiliation NTT Service Integration Laboratories()
2nd Author's Name Tatsuaki KIMURA
2nd Author's Affiliation NTT Service Integration Laboratories
3rd Author's Name Yasuhiro IKEDA
3rd Author's Affiliation NTT Service Integration Laboratories
4th Author's Name Noriaki KAMIYAMA
4th Author's Affiliation NTT Service Integration Laboratories
5th Author's Name Ryoichi KAWAHARA
5th Author's Affiliation NTT Service Integration Laboratories
Date 2010-11-18
Paper # CQ2010-49
Volume (vol) vol.110
Number (no) 287
Page pp.pp.-
#Pages 6
Date of Issue