https://scholars.lib.ntu.edu.tw/handle/123456789/634889
標題: | Sample Complexity of Kernel-Based Q-Learning | 作者: | Yeh, Sing Yuan Chang, Fu Chieh Yueh, Chang Wei PEI-YUAN WU Bernacchia, Alberto Vakili, Sattar |
公開日期: | 1-一月-2023 | 卷: | 206 | 來源出版物: | Proceedings of Machine Learning Research | 摘要: | Modern reinforcement learning (RL) often faces an enormous state-action space. Existing analytical results are typically for settings with a small number of state-actions, or simple models such as linearly modeled Q-functions. To derive statistically efficient RL policies handling large state-action spaces, with more general Q-functions, some recent works have considered nonlinear function approximation using kernel ridge regression. In this work, we derive sample complexities for kernel based Q-learning when a generative model exists. We propose a nonparametric Q-learning algorithm which finds an ϵ-optimal policy in an arbitrarily large scale discounted MDP. The sample complexity of the proposed algorithm is order optimal with respect to ϵ and the complexity of the kernel (in terms of its information gain). To the best of our knowledge, this is the first result showing a finite sample complexity under such a general model. |
URI: | https://scholars.lib.ntu.edu.tw/handle/123456789/634889 |
顯示於: | 電機工程學系 |
在 IR 系統中的文件,除了特別指名其著作權條款之外,均受到著作權保護,並且保留所有的權利。