English   español  
Please use this identifier to cite or link to this item: http://hdl.handle.net/10261/167040
Share/Impact:
Statistics
logo share SHARE logo core CORE   Add this article to your Mendeley library MendeleyBASE

Visualizar otros formatos: MARC | Dublin Core | RDF | ORE | MODS | METS | DIDL | DATACITE
Exportar a otros formatos:

Title

Dual REPS: A generalization of relative entropy policy search exploiting bad experiences

AuthorsColomé, Adrià ; Torras, Carme
Issue Date2017
PublisherInstitute of Electrical and Electronics Engineers
CitationIEEE Transactions on Robotics 33(4): 978-985 (2017)
AbstractPolicy search (PS) algorithms are widely used for their simplicity and effectiveness in finding solutions for robotic problems. However, most current PS algorithms derive policies by statistically fitting the data from the best experiments only. This means that experiments yielding a poor performance are usually discarded or given too little influence on the policy update. In this paper, we propose a generalization of the relative entropy policy search (REPS) algorithm that takes bad experiences into consideration when computing a policy. The proposed approach, named dual REPS (DREPS) following the philosophical interpretation of the duality between good and bad, finds clusters of experimental data yielding a poor behavior and adds them to the optimization problem as a repulsive constraint. Thus, considering that there is a duality between good and bad data samples, both are taken into account in the stochastic search for a policy. Additionally, a cluster with the best samples may be included as an attractor to enforce faster convergence to a single optimal solution in multimodal problems. We first tested our proposed approach in a simulated reinforcement learning setting and found that DREPS considerably speeds up the learning process, especially during the early optimization steps and in cases where other approaches get trapped in between several alternative maxima. Further experiments in which a real robot had to learn a task with a multimodal reward function confirm the advantages of our proposed approach with respect to REPS.
Publisher version (URL)https://doi.org/10.1109/TRO.2017.2679202
URIhttp://hdl.handle.net/10261/167040
DOIhttp://dx.doi.org/10.1109/TRO.2017.2679202
Identifiersdoi: 10.1109/TRO.2017.2679202
issn: 1552-3098
e-issn: 1941-0468
Appears in Collections:(IRII) Artículos
Files in This Item:
File Description SizeFormat 
DualREPS.pdf1,77 MBAdobe PDFThumbnail
View/Open
Show full item record
Review this work
 


WARNING: Items in Digital.CSIC are protected by copyright, with all rights reserved, unless otherwise indicated.