| IEICE Technical Committee Submission System Conference Paper's Information |
Online Proceedings [Sign in] Tech. Rep. Archives |
| Go Top Page | Go Previous | [Japanese] / [English] |
| Paper Abstract and Keywords | ||
| Presentation | 2018-03-01 10:50 Remote replication for LHD's large data -- 4 Gbps data transfer of 145 TB data using MMCFTP and piplelined replication -- Kenjiro Yamanaka (SOKENDAI/NII), Hideya Nakanishi (NIFS), Takahisa Ozeki (QST), Shunji ABE, Shigeo Urushidani (SOKENDAI/NII), Takashi Yamamoto, Masahiko Emoto, Noriyoshi Nakajima (NIFS)  NS2017-180 |
|
| Abstract | (in Japanese) | (See Japanese page) |
| (in English) | Large volumes of data shared internationally in advanced research projects are often composed of many small files. Transfer efficiency is low when file transfer is performed many times while keeping a small file. Although transfer efficiency is improved when archiving and transferring files, there is a problem that it takes extra time for archiving / decompression, storage space for storing archive files is required separately. This point was also an issue in the experiment of remotely replicating measurement data of NIFS’s Large Helical Device (LHD) to the ITER remote experiment centre. Therefore, about 145 TB of data to be transferred was divided into about 13000 based on the LHD experiment number, and archiving and file transfer were pipelined. Since archive creation and transfer are done in parallel, archiving time does not affect throughput. Also, since the archive file is erased after the transfer, large storage is unnecessary. This storage requires two times the transfer speed at the same time as reading and writing, making it easier to become a bottleneck. Currently, the main constraint of high-speed replication is in storage performance, not in communication distance. | |
| Keyword | (in Japanese) | (See Japanese page) |
| (in English) | MMCFTP / Large Helical Device (LHD) / The lot of small files (LOSF) problem / Remote replication / Pipelining / / / | |
| Reference Info. | IEICE Tech. Rep., vol. 117, no. 459, NS2017-180, pp. 73-78, March 2018. | |
| Paper # | NS2017-180 | |
| Date of Issue | 2018-02-22 (NS) | |
| ISSN | Print edition: ISSN 0913-5685 Online edition: ISSN 2432-6380 | |
| Copyright and reproduction |
All rights are reserved and no part of this publication may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopy, recording, or any information storage and retrieval system, without permission in writing from the publisher. Notwithstanding, instructors are permitted to photocopy isolated articles for noncommercial classroom use without fee. (License No.: 10GA0019/12GB0052/13GB0056/17GB0034/18GB0034) | |
| Download PDF | NS2017-180 |
|
| Conference Information | |
| Committee | NS IN |
| Conference Date | 2018-03-01 - 2018-03-02 |
| Place (in Japanese) | (See Japanese page) |
| Place (in English) | Phoenix Seagaia Resort |
| Topics (in Japanese) | (See Japanese page) |
| Topics (in English) | General |
| Paper Information | |
| Registration To | NS |
| Conference Code | 2018-03-NS-IN |
| Language | Japanese |
| Title (in Japanese) | (See Japanese page) |
| Sub Title (in Japanese) | (See Japanese page) |
| Title (in English) | Remote replication for LHD's large data |
| Sub Title (in English) | 4 Gbps data transfer of 145 TB data using MMCFTP and piplelined replication |
| Keyword(1) | MMCFTP |
| Keyword(2) | Large Helical Device (LHD) |
| Keyword(3) | The lot of small files (LOSF) problem |
| Keyword(4) | Remote replication |
| Keyword(5) | Pipelining |
| Keyword(6) | |
| Keyword(7) | |
| Keyword(8) | |
| 1st Author's Name | Kenjiro Yamanaka |
| 1st Author's Affiliation | SOKENDAI/NII (SOKENDAI/NII) |
| 2nd Author's Name | Hideya Nakanishi |
| 2nd Author's Affiliation | National Institute for Fusion Science (NIFS) |
| 3rd Author's Name | Takahisa Ozeki |
| 3rd Author's Affiliation | National Institutes for Quantum and Radiological Science and Technology (QST) |
| 4th Author's Name | Shunji ABE |
| 4th Author's Affiliation | SOKENDAI/NII (SOKENDAI/NII) |
| 5th Author's Name | Shigeo Urushidani |
| 5th Author's Affiliation | SOKENDAI/NII (SOKENDAI/NII) |
| 6th Author's Name | Takashi Yamamoto |
| 6th Author's Affiliation | National Institute for Fusion Science (NIFS) |
| 7th Author's Name | Masahiko Emoto |
| 7th Author's Affiliation | National Institute for Fusion Science (NIFS) |
| 8th Author's Name | Noriyoshi Nakajima |
| 8th Author's Affiliation | National Institute for Fusion Science (NIFS) |
| 9th Author's Name | |
| 9th Author's Affiliation | () |
| 10th Author's Name | |
| 10th Author's Affiliation | () |
| 11th Author's Name | |
| 11th Author's Affiliation | () |
| 12th Author's Name | |
| 12th Author's Affiliation | () |
| 13th Author's Name | |
| 13th Author's Affiliation | () |
| 14th Author's Name | |
| 14th Author's Affiliation | () |
| 15th Author's Name | |
| 15th Author's Affiliation | () |
| 16th Author's Name | |
| 16th Author's Affiliation | () |
| 17th Author's Name | |
| 17th Author's Affiliation | () |
| 18th Author's Name | |
| 18th Author's Affiliation | () |
| Speaker | Author-1 |
| Date Time | 2018-03-01 10:50:00 |
| Presentation Time | 20 minutes |
| Registration for | NS |
| Paper # | NS2017-180 |
| Volume (vol) | vol.117 |
| Number (no) | no.459 |
| Page | pp.73-78 |
| #Pages | 6 |
| Date of Issue | 2018-02-22 (NS) |
[Return to Top Page]