Modularization of End-to-End Learning: Case Study in Arcade Games

Melnik A, Fleer S, Schilling M, Ritter H (2019)
In: 32nd Conference on Neural Information Processing Systems (NeurIPS 2018), Workshop on Causal Learning.

Konferenzbeitrag | Englisch
 
Download
Es wurden keine Dateien hochgeladen. Nur Publikationsnachweis!
Abstract / Bemerkung
Complex environments and tasks pose a difficult problem for holistic end-to-end learning approaches. Decomposition of an environment into interacting controllable and non-controllable objects allows supervised learning for non-controllable objects and universal value function approximator learning for controllable objects. Such decomposition should lead to a shorter learning time and better generalisation capability. Here, we consider arcade-game environments as sets of interacting objects (controllable, non-controllable) and propose a set of functional modules that are specialized on mastering different types of interactions in a broad range of environments. The modules utilize regression, supervised learning, and reinforcement learning algorithms. Results of this case study in different Atari games suggest that human-level performance can be achieved by a learning agent within a human amount of game experience (10-15 minutes game time) when a proper decomposition of an environment or a task is provided. However, automatization of such decomposition remains a challenging problem. This case study shows how a model of a causal structure underlying an environment or a task can benefit learning time and generalization capability of the agent, and argues in favor of exploiting modular structure in contrast to using pure end-to-end learning approaches.
Stichworte
Arcade Games; Causal Learning; Hierarchical Reinforcement Learning; End-to-End Learning
Erscheinungsjahr
2019
Titel des Konferenzbandes
32nd Conference on Neural Information Processing Systems (NeurIPS 2018), Workshop on Causal Learning
Konferenz
32nd Conference on Neural Information Processing Systems (NeurIPS 2018)
Konferenzort
Montréal, Canada.
Konferenzdatum
2018-12-02 – 2018-12-08
Page URI
https://pub.uni-bielefeld.de/record/2933988

Zitieren

Melnik A, Fleer S, Schilling M, Ritter H. Modularization of End-to-End Learning: Case Study in Arcade Games. In: 32nd Conference on Neural Information Processing Systems (NeurIPS 2018), Workshop on Causal Learning. 2019.
Melnik, A., Fleer, S., Schilling, M., & Ritter, H. (2019). Modularization of End-to-End Learning: Case Study in Arcade Games. 32nd Conference on Neural Information Processing Systems (NeurIPS 2018), Workshop on Causal Learning
Melnik, A., Fleer, S., Schilling, M., and Ritter, H. (2019). “Modularization of End-to-End Learning: Case Study in Arcade Games” in 32nd Conference on Neural Information Processing Systems (NeurIPS 2018), Workshop on Causal Learning.
Melnik, A., et al., 2019. Modularization of End-to-End Learning: Case Study in Arcade Games. In 32nd Conference on Neural Information Processing Systems (NeurIPS 2018), Workshop on Causal Learning.
A. Melnik, et al., “Modularization of End-to-End Learning: Case Study in Arcade Games”, 32nd Conference on Neural Information Processing Systems (NeurIPS 2018), Workshop on Causal Learning, 2019.
Melnik, A., Fleer, S., Schilling, M., Ritter, H.: Modularization of End-to-End Learning: Case Study in Arcade Games. 32nd Conference on Neural Information Processing Systems (NeurIPS 2018), Workshop on Causal Learning. (2019).
Melnik, Andrew, Fleer, Sascha, Schilling, Malte, and Ritter, Helge. “Modularization of End-to-End Learning: Case Study in Arcade Games”. 32nd Conference on Neural Information Processing Systems (NeurIPS 2018), Workshop on Causal Learning. 2019.
Externes Material:
In sonstiger Relation

Export

Markieren/ Markierung löschen
Markierte Publikationen

Open Data PUB

Quellen

arXiv: 1901.09895

Suchen in

Google Scholar