English   español  
Please use this identifier to cite or link to this item: http://hdl.handle.net/10261/133994
logo share SHARE   Add this article to your Mendeley library MendeleyBASE
Visualizar otros formatos: MARC | Dublin Core | RDF | ORE | MODS | METS | DIDL
Exportar a otros formatos:

Competitive function approximation for reinforcement learning

AuthorsAgostini, Alejandro ; Celaya, Enric
Issue Date2014
PublisherCSIC-UPC - Instituto de Robótica e Informática Industrial (IRII)
CitationIRI-TR-14-05 (2014)
AbstractThe application of reinforcement learning to problems with continuous domains requires representing the value function by means of function approximation. We identify two aspects of reinforcement learning that make the function approximation process hard: non-stationarity of the target function and biased sampling. Non-stationarity is the result of the bootstrapping nature of dynamic programming where the value function is estimated using its current approximation. Biased sampling occurs when some regions of the state space are visited too often, causing a reiterated updating with similar values which fade out the occasional updates of infrequently sampled regions. We propose a competitive approach for function approximation where many different local approximators are available at a given input and the one with expectedly best approximation is selected by means of a relevance function. The local nature of the approximators allows their fast adaptation to non-stationary changes and mitigates the biased sampling problem. The coexistence of multiple approximators updated and tried in parallel permits obtaining a good estimation much faster than would be possible with a single approximator. Experiments in different benchmark problems show that the competitive strategy provides a faster and more stable learning than non-competitive approaches.
Publisher version (URL)http://www.iri.upc.edu/publications/show/1599
Appears in Collections:(IRII) Informes y documentos de trabajo
Files in This Item:
File Description SizeFormat 
Reinforcement-Learning.pdf4,45 MBAdobe PDFThumbnail
Show full item record
Review this work

WARNING: Items in Digital.CSIC are protected by copyright, with all rights reserved, unless otherwise indicated.