Artificial Intelligence (AI) will be the dominant technology of the future and will impact every corner of society. In particular, AI / ML (machine learning) will shape how communication networks, a lifeline of our society, will be run. Many companies in the ICT sector are exploring how to make best use of AI/ML. ITU has been at the forefront of this endeavour exploring how to best apply AI/ML in future networks including 5G networks. The time is therefore right to bring together the technical community and stakeholders to brainstorm, innovate and solve relevant problems in 5G using AI/ML. Building on its standards work, ITU is conducting a global ITU AI/ML 5G Challenge on the theme “How to apply ITU's ML architecture in 5G networks".
Participants will be able to solve real world problems, based on standardized technologies developed for ML in 5G networks. Teams will be required to enable, create, train and deploy ML models (such that participants will acquire hands-on experience in AI/ML in areas relevant to 5G). Participation is open to ITU Member States, Sector Members, Associates and Academic Institutions and to any individual from a country that is a member of ITU.
Theme 1 from KDDI
Analysis on Route Information Failure in IP Core Networks by NFV-Based Test Environment
Challenge Problem
The stable and high quality Internet connectivity is mandatory to 5G mobile networks, but once something unexpected happens, the influence of the defect is quite severing. In addition, the Internet is operated mutually among operators, and one failure in a domain happens to be rapidly spread all over the world. Only highly experienced operators can tackle such globally affected network failure and the anomaly detection is desired to be automatically and rapidly performed by AI/ML. Each mobile operator has at least one IP backbone network which is attached to mobile core networks. The IP backbone network interconnect with other operators’ backbone networks via border gateway routers. Border gateway routers continuously update their route information from received internal/external route information, and need to feed back and forth them appropriately. Thus, those routers play a very significant role for 5G services, and the defect in hardware/software as well as mis-operation is desired to be immediately detected to maintain a certain service level.
In this problem, the data sets at border gateway routers are provided for this problem along with network status information such as normal, a failure, mis-operation and so forth, as normal/abnormal labels. Participants are required to create the model to pinpoint the network status of failures and mis-operation using those data sets and evaluate the performance of the developed model.
Participants must create and train a model of AI/ML by the data set for learning and verify the performance of the derived model by the data set of evaluation in terms of anomaly detection and root cause analysis.
The data sets used for this challenge were created in the NFV-based test environment simulated for a commercial IP core network according to [1-1]. In this sense, they are synthetic data, but as similar as the real data, resulting from our NFV-based test environment. The data sets consist of normal/abnormal labels, performance monitoring data sets such as traffic volume and CPU/MEM usage ratio, and route information such as Border Gateway Protocols (BGP) static metrics as well as BGP route information. Whilst the data sets were kept to be stored for a long period enough to be analysed, intentional network failures were applied to the network, leading to abnormal labels.
In order create data sets, the data collector was developed to collects and stores data sets every minute from the network. Once a failure is intentionally caused and recovered, the network indicates a failure or normal status after a period of transition, corresponding to failure data (orange arrows) and recovery data (blue arrows). The period of transition depends on a failure scenario and enough guard time is desired to be considered. The time interval between a failure and a recovery is 5 min (Fig. 1-1).
Two types of dataset files for learning and evaluation are provided to participants (Fig. 1-2). The dataset files for learning can be used for training AI models, and the dataset files for evaluation can be used for evaluating performance of the trained model. The dataset files for learning corresponds to use cases when all failure scenarios are comprehensively invoked at all possible failure points. The dataset files for evaluation corresponds to the case when a combination of a failure scenario and a failure point is randomly and limitedly generated.
[1-1] J. Kawasaki, G. Mouri, and Y. Suzuki, "Comparative Analysis of Network Fault Classification Using Machine Learning," NOMS2020, 10.1109/NOMS47738.2020.9110454.
Create and train a model of AI/ML by the data set for learning and verify the performance of the derived model by the data set of evaluation in terms of anomaly detection and root cause analysis.
Participants must submit a power point file with a pdf format indicating the results and a demonstration video showing predicting performance.
Please send your result to the following address by E-mail.
Subject of E-mail has to be [ITU-JP-Theme1].
Theme 2 from NEC
Network State Estimation by Analyzing Raw Video Data
Due to COVID-19 pandemic, the importance of interactive live video streaming services, e.g., telework system using web cameras, has been increasing. However, the Internet cannot avoid to accommodate the increasing traffic generated from such bandwidth-consuming video streaming services, which results in heavy congestion. In case of video streaming services by over-the-top (OTT) service providers, e.g., Netflix, YouTube, and Amazon, they address the issue in COVID-19 pandemic by setting lower standard resolution based on traffic load of their services. Similarly, in case of interactive video streaming services using web cameras, video quality should be optimized based on their network state. This situation causes a challenging issue of passive network state estimation by analyzing raw video data. Conventionally, many researchers in the field of video streaming have addressed to estimate network state by using playback buffer state. However, analyzing not KPI, e.g., bit rate and resolution, but raw video images are important for practical use cases such as telework system. Recently, we observe a new trend of artificial intelligence (AI) techniques, such as deep learning, that make a breakthrough of raw image analysis. This challenge is the first step to understand relationship between raw video images and network state.
RTP [2-1], a communication protocol suitable for live video streaming services using web cameras, is used here. Video image quality, e.g., noise, depends on the network condition (Fig. 2-1).
[2-1] H. Schulzrinne, et al., ``RTP: A Transport Protocol for Real-Time Applications," Request for Comments 3550, July 2003, https://tools.ietf.org/html/rfc3550.
The goal of this challenge is to estimate network state, i.e., throughput and loss ratio, from given raw video data sets. The participants are expected to train and test an AI model using the video data with labels of network state (Fig. 2-2).
All submission will be evaluated in terms of a performance measure (MAE) and technical excellence. Mean absolute error ($MAE$) will be used as a measure, which is defined as follows. From the number of tests is $n$ and the $i$th estimation value is $Estimation[i]$, $MAE$ is calculated for each of bandwidth and loss ratio. \[ MAE = \frac{1}{n}\sum_{i=1}^{n} | Estimation[i] - Answer | \]
Two types of videos are provided. 1) Original video We use open data as an original video. The original video follows .mp4 format (YouTube-8M).
2) Received video The received videos are also formatted by .mp4. In addition, file name of a video delivered in a network condition of certain bandwidth and loss ratio follows "videoid_bandwidth_loss.mp4". Datasets are generated in our lab environment (Fig. 2-3). Video Streamer (VS) transmits original video to Video Viewer (VV) via Network Emulator (NE) over RTP. Setting parameters are shown in Table 2-1.
NE control traffic rate and packet loss based on the following policy. Video traffic is shaped with predefined throughput and packets will be lost with predefined loss ratio. Dataset for training is generated on the basis of the following network condition. - Traffic rate: from 1100kbps to 2000kbps at 100kbps intervals - Packet loss ratio: 0.001%, 0.01%, 0.025%, 0.05%, 0.1%
Participants need to submit report.
-Report includes explanation of your method/approach, evaluation results (MAE, See "Evaluation criteria") for provided data set, and consideration at least.
-Report format: A4 size, pdf, 4 pages at most.
Please send your result to the following address by E-mail.
Subject of E-mail has to be [ITU-JP-Theme2].
Theme 3 from RISING
WiFi Location Estimation
Please see Japanese page. This theme is not for final conference of ITU.
Akihiro Nakao(Chair)
Univ.of Tokyo
Takeo Fujii
Univ. of Electro-Comm.
Yoichi Maeda
TTC
Tomohiro Otani
KDDI Research, Inc.
Hideyuki Shimonishi
NEC Corp.
Takuji Tachibana
Univ. of Fukui
Kazuya Tsukamoto
Kyushu Inst. of Tech.
Koichi Adachi
Univ. of Electro-Comm.
Daisuke Anzai
Nagoya Inst. of Tech.
Kazunori Hayashi
Kyoto Univ.
Yusuke Hirota
NICT
Suguru Kameda
Tohoku Univ.
Ryota Kawashima
Nagoya Inst. of Tech.
Tatsuaki Kimura
Osaka Univ.
Kazuhiko Kinoshita
Tokushima Univ.
Yoshiaki Narusue
Univ. of Tokyo
Daiki Nobayashi
Kyushu Inst. of Tech.
Masahiro Sasabe
NAIST
Ryoichi Shinkuma
Kyoto Univ.
Yusuke Shinohara
NEC
Osamu Takyu
Shinshu Univ.
Yuya Tarutani
Okayama Univ.
Daisuke Umehara
Kyoto Inst. of Tech.
Hiroshi Yamamoto
Ritsumeikan Univ.
Koji Yamamoto
Kyoto Univ.
Shinji Yamashita
Fujitsu Lab.