Using cases as heuristics in reinforcement learning: A transfer learning application
Nenhuma Miniatura disponível
Citações na Scopus
25
Tipo de produção
Artigo de evento
Data
2011-07-02
Autores
CELIBERTO JUNIOR, L. A.
MATSUURA, J. P.
DE MANTARAS, R. L.
Reinaldo Bianchi
MATSUURA, J. P.
DE MANTARAS, R. L.
Reinaldo Bianchi
Orientador
Periódico
IJCAI International Joint Conference on Artificial Intelligence
Título da Revista
ISSN da Revista
Título de Volume
Citação
CELIBERTO JUNIOR, L. A.; MATSUURA, J. P.; DE MANTARAS, R. L.; BIANCHI, R. Using cases as heuristics in reinforcement learning: A transfer learning application. IJCAI International Joint Conference on Artificial Intelligence. p. 1211-1217, July, 2011.
Texto completo (DOI)
Palavras-chave
Resumo
In this paper we propose to combine three AI techniques to speed up a Reinforcement Learning algorithm in a Transfer Learning problem: Case-based Reasoning, Heuristically Accelerated Reinforcement Learning and Neural Networks. To do so, we propose a new algorithm, called L3, which works in 3 stages: in the first stage, it uses Reinforcement Learning to learn how to perform one task, and stores the optimal policy for this problem as a case-base; in the second stage, it uses a Neural Network to map actions from one domain to actions in the other domain and; in the third stage, it uses the case-base learned in the first stage as heuristics to speed up the learning performance in a related, but different, task. The RL algorithm used in the first phase is the Q-learning and in the third phase is the recently proposed Case-based Heuristically Accelerated Q-learning. A set of empirical evaluations were conducted in transferring the learning between two domains, the Acrobot and the Robocup 3D: the policy learned during the solution of the Acrobot Problem is transferred and used to speed up the learning of stability policies for a humanoid robot in the Robocup 3D simulator. The results show that the use of this algorithm can lead to a significant improvement in the performance of the agent.