| Presentation | 2002/3/11 Ridge Parameter Determination in Infinite Dimensional Hypothesis Spaces Masashi SUGIYAMA, Klaus-Robert MULLER, |
|---|---|
| PDF Download Page | PDF download Page Link |
| Abstract(in Japanese) | (See Japanese page) |
| Abstract(in English) | We discuss the problem of determining ridge parameter in the context of kernel regression. Previously, a generalization error estimator called the subspace information criterion (SIC) was proposed, that could be successfully applied to determining the ridge parameter. SIC is an unbiased estimator of the generalization error for the finite sample case under the conditions that the learning target function belongs to a specified reproducing kernel Hubert space (RKHS) H and the kernel regression model spans the whole space H. These conditions hold only if dim H ≦ M where M (<∽) is the number of training samples. Therefore, SIC could be applied only to finite dimensional RKHSs. In this paper, we extend the range of applicability of SIC, and show that even if the kernel regression model does not span the whole space H, SIC is an unbiased estimator of an essential part of the generalization error. Our extension allows us to make use of any RKHSs including infinite dimensional ones. We further show that when the kernel matrix is invertible, SIC can be expressed in a much simpler form, making its computation highly efficient. Finally, we show by computer simulations with real and artificial data sets that the extended SIC works well in ridge parameter selection. |
| Keyword(in Japanese) | (See Japanese page) |
| Keyword(in English) | generalization error / model selection / subspace information criterion / kernel regression / reproducing kernel Hubert space / cross-validation |
| Paper # | NC2001-135 |
| Date of Issue |
| Conference Information | |
| Committee | NC |
|---|---|
| Conference Date | 2002/3/11(1days) |
| Place (in Japanese) | (See Japanese page) |
| Place (in English) | |
| Topics (in Japanese) | (See Japanese page) |
| Topics (in English) | |
| Chair | |
| Vice Chair | |
| Secretary | |
| Assistant | |
| Paper Information | |
| Registration To | Neurocomputing (NC) |
|---|---|
| Language | ENG |
| Title (in Japanese) | (See Japanese page) |
| Sub Title (in Japanese) | (See Japanese page) |
| Title (in English) | Ridge Parameter Determination in Infinite Dimensional Hypothesis Spaces |
| Sub Title (in English) | |
| Keyword(1) | generalization error |
| Keyword(2) | model selection |
| Keyword(3) | subspace information criterion |
| Keyword(4) | kernel regression |
| Keyword(5) | reproducing kernel Hubert space |
| Keyword(6) | cross-validation |
| 1st Author's Name | Masashi SUGIYAMA |
| 1st Author's Affiliation | Department of Computer Science, Tokyo Institute of Technology() |
| 2nd Author's Name | Klaus-Robert MULLER |
| 2nd Author's Affiliation | Fraunhofer FIRST, IDA:Department of Computer Science,University of Potsudam |
| Date | 2002/3/11 |
| Paper # | NC2001-135 |
| Volume (vol) | vol.101 |
| Number (no) | 735 |
| Page | pp.pp.- |
| #Pages | 8 |
| Date of Issue | |