| Presentation | 2014-11-18 Robust Estimation under Heavy Contamination using Unnormalized Models Takafumi Kanamori, Hironori Fujisawa, |
|---|---|
| PDF Download Page |  PDF download Page Link |
| Abstract(in Japanese) | (See Japanese page) |
| Abstract(in English) | In data analysis, contamination caused by outliers is inevitable, and robust statistical methods are strongly demanded. In this paper, our concern is to develop a new approach for robust data analysis on the basis of scoring rules. The scoring rule is a discrepancy measure to assess the quality of probabilistic forecasts. We propose a simple method of estimating not only parameters in the statistical model but also the contamination ratio of outliers. Estimating the contamination ratio is important, since one can detect the outliers in training samples based on the estimated contamination ratio. For this purpose, we use scoring rules with extended statistical models called unnormalized models. Also, regression problems are considered. We study complex heterogeneous contamination wherein the contamination ratio of outliers in a dependent variable may depend on independent variables. We propose a simple method to obtain a robust regression estimator under heterogeneous contamination. In addition, our method provides an estimator of the expected contamination ratio that is available to detect the outliers in training samples. Numerical experiments demonstrate the effectiveness of our method compared to conventional estimators. |
| Keyword(in Japanese) | (See Japanese page) |
| Keyword(in English) | Scoring Rules / Unnormalized Models / Contamination Ratio / Regression / Heterogeneous Contamination |
| Paper # | IBISML2014-68 |
| Date of Issue |
| Conference Information | |
| Committee | IBISML |
|---|---|
| Conference Date | 2014/11/10(1days) |
| Place (in Japanese) | (See Japanese page) |
| Place (in English) | |
| Topics (in Japanese) | (See Japanese page) |
| Topics (in English) | |
| Chair | |
| Vice Chair | |
| Secretary | |
| Assistant | |
| Paper Information | |
| Registration To | Information-Based Induction Sciences and Machine Learning (IBISML) |
|---|---|
| Language | ENG |
| Title (in Japanese) | (See Japanese page) |
| Sub Title (in Japanese) | (See Japanese page) |
| Title (in English) | Robust Estimation under Heavy Contamination using Unnormalized Models |
| Sub Title (in English) | |
| Keyword(1) | Scoring Rules |
| Keyword(2) | Unnormalized Models |
| Keyword(3) | Contamination Ratio |
| Keyword(4) | Regression |
| Keyword(5) | Heterogeneous Contamination |
| 1st Author's Name | Takafumi Kanamori |
| 1st Author's Affiliation | Nagoya University() |
| 2nd Author's Name | Hironori Fujisawa |
| 2nd Author's Affiliation | The Institute of Statistical Mathematics |
| Date | 2014-11-18 |
| Paper # | IBISML2014-68 |
| Volume (vol) | vol.114 |
| Number (no) | 306 |
| Page | pp.pp.- |
| #Pages | 8 |
| Date of Issue | |