ITU
AI/ML in 5G Challenge

The stable and high quality Internet connectivity is mandatory to 5G mobile networks, but once something unexpected happens, the influence of the defect is quite severing. In addition, the Internet is operated mutually among operators, and one failure in a domain happens to be rapidly spread all over the world. Only highly experienced operators can tackle such globally affected network failure and the anomaly detection is desired to be automatically and rapidly performed by AI/ML. Each mobile operator has at least one IP backbone network which is attached to mobile core networks. The IP backbone network interconnect with other operators’ backbone networks via border gateway routers. Border gateway routers continuously update their route information from received internal/external route information, and need to feed back and forth them appropriately. Thus, those routers play a very significant role for 5G services, and the defect in hardware/software as well as mis-operation is desired to be immediately detected to maintain a certain service level.

In this problem, the data sets at border gateway routers are provided for this problem along with network status information such as normal, a failure, mis-operation and so forth, as normal/abnormal labels. Participants are required to create the model to pinpoint the network status of failures and mis-operation using those data sets and evaluate the performance of the developed model.

Participants must create and train a model of AI/ML by the data set for learning and verify the performance of the derived model by the data set of evaluation in terms of anomaly detection and root cause analysis.

Create and train a model of AI/ML by the data set for learning and verify the performance of the derived model by the data set of evaluation in terms of anomaly detection and root cause analysis. Participants must submit a power point file with a pdf format indicating the results and a demonstration video showing predicting performance. Please send your result to the following address by E-mail.

Subject of E-mail has to be [ITU-JP-Theme1].

Due to COVID-19 pandemic, the importance of interactive live video streaming services, e.g., telework system using web cameras, has been increasing. However, the Internet cannot avoid to accommodate the increasing traffic generated from such bandwidth-consuming video streaming services, which results in heavy congestion. In case of video streaming services by over-the-top (OTT) service providers, e.g., Netflix, YouTube, and Amazon, they address the issue in COVID-19 pandemic by setting lower standard resolution based on traffic load of their services. Similarly, in case of interactive video streaming services using web cameras, video quality should be optimized based on their network state.
This situation causes a challenging issue of passive network state estimation by analyzing raw video data. Conventionally, many researchers in the field of video streaming have addressed to estimate network state by using playback buffer state. However, analyzing not KPI, e.g., bit rate and resolution, but raw video images are important for practical use cases such as telework system. Recently, we observe a new trend of artificial intelligence (AI) techniques, such as deep learning, that make a breakthrough of raw image analysis. This challenge is the first step to understand relationship between raw video images and network state.

RTP [2-1], a communication protocol suitable for live video streaming services using web cameras, is used here. Video image quality, e.g., noise, depends on the network condition (Fig. 2-1).

[2-1] H. Schulzrinne, et al., ``RTP: A Transport Protocol for Real-Time Applications," Request for Comments 3550, July 2003, https://tools.ietf.org/html/rfc3550.

The goal of this challenge is to estimate network state, i.e., throughput and loss ratio, from given raw video data sets. The participants are expected to train and test an AI model using the video data with labels of network state (Fig. 2-2).

All submission will be evaluated in terms of a performance measure (MAE) and technical excellence.

Mean absolute error ($MAE$) will be used as a measure, which is defined as follows. From the number of tests is $n$ and the $i$th estimation value is $Estimation[i]$, $MAE$ is calculated for each of bandwidth and loss ratio. \[ MAE = \frac{1}{n}\sum_{i=1}^{n} | Estimation[i] - Answer | \]

Participants need to submit report.

-Report includes explanation of your method/approach, evaluation results (MAE, See "Evaluation criteria") for provided data set, and consideration at least.

-Report format: A4 size, pdf, 4 pages at most.

Please send your result to the following address by E-mail.

Subject of E-mail has to be [ITU-JP-Theme2].

Please see Japanese page. This theme is not for final conference of ITU.