| Presentation | 2000/7/11 NC2000-49 Robust Reinforcement Learning Jun Morimoto, Kenji Doya, |
|---|---|
| PDF Download Page | PDF download Page Link |
| Abstract(in Japanese) | (See Japanese page) |
| Abstract(in English) | This paper proposes a new reinforcement learning(RL)paradigm that explicitly takes into account input disturbance as well as modeling errors. The use of environmental models in RL is quite popular for both off-line learning by simulations and for on-line action planning. However, the difference between the model and the real environment can lead to unpredictable, often unwanted results. Based on the theory of H^∞ control, we consider a differential game in which a'disturbing'agent tries to make the worst possible disturbance while a'control'agent tries to make the best control input. The problem is formulated as finding a min-max solution of a value function that takes into account the changes in the reward due to the disturbance and the amplitude of the disturbance. We derive on-line learning algorithms for estimating the value function and for calculating both the worst disturbance and the best control in reference to the value function. We tested the paradigm, which we call"Robust Reinforcement Learning(RRL), "in the task of inverted pendulum. The control by RRL achieved robust performance against two-fold changes in the pendulum length while a standard RL control could not deal with such environmental changes. |
| Keyword(in Japanese) | (See Japanese page) |
| Keyword(in English) | reinforcement learning / robust control / H-infinity |
| Paper # | NC2000-49 |
| Date of Issue |
| Conference Information | |
| Committee | NC |
|---|---|
| Conference Date | 2000/7/11(1days) |
| Place (in Japanese) | (See Japanese page) |
| Place (in English) | |
| Topics (in Japanese) | (See Japanese page) |
| Topics (in English) | |
| Chair | |
| Vice Chair | |
| Secretary | |
| Assistant | |
| Paper Information | |
| Registration To | Neurocomputing (NC) |
|---|---|
| Language | ENG |
| Title (in Japanese) | (See Japanese page) |
| Sub Title (in Japanese) | (See Japanese page) |
| Title (in English) | NC2000-49 Robust Reinforcement Learning |
| Sub Title (in English) | |
| Keyword(1) | reinforcement learning |
| Keyword(2) | robust control |
| Keyword(3) | H-infinity |
| 1st Author's Name | Jun Morimoto |
| 1st Author's Affiliation | Graduate School of Information Science, Nara Institute of Science and Technology:Kawato Dynamic Brain Project, ERATO JST() |
| 2nd Author's Name | Kenji Doya |
| 2nd Author's Affiliation | Information Sciences Devision, ATR International:CREST, JST:Graduate School of Information Science, Nara Institute of Science and Technology |
| Date | 2000/7/11 |
| Paper # | NC2000-49 |
| Volume (vol) | vol.100 |
| Number (no) | 191 |
| Page | pp.pp.- |
| #Pages | 8 |
| Date of Issue | |