Publications
For news about publications, follow us on X/Twitter: 
Click on any author names or tags to filter publications.
All topic tags:
surveydeep-rlmulti-agent-rlagent-modellingad-hoc-teamworkautonomous-drivinggoal-recognitionexplainable-aicausalgeneralisationsecurityemergent-communicationiterated-learningintrinsic-rewardsimulatorstate-estimationdeep-learningtransfer-learning
Selected tags (click to remove):
deep-rlStefano-V.-AlbrechtTrevor-McInroeTMLR
2024
Trevor McInroe, Lukas Schäfer, Stefano V. Albrecht
Multi-Horizon Representations with Hierarchical Forward Models for Reinforcement Learning
Transactions on Machine Learning Research, 2024
Abstract | BibTex | arXiv | Code
TMLRdeep-rl
Abstract:
Learning control from pixels is difficult for reinforcement learning (RL) agents because representation learning and policy learning are intertwined. Previous approaches remedy this issue with auxiliary representation learning tasks, but they either do not consider the temporal aspect of the problem or only consider single-step transitions. Instead, we propose Hierarchical k-Step Latent (HKSL), an auxiliary task that learns representations via a hierarchy of forward models that operate at varying magnitudes of step skipping while also learning to communicate between levels in the hierarchy. We evaluate HKSL in a suite of 30 robotic control tasks and find that HKSL either reaches higher episodic returns or converges to maximum performance more quickly than several current baselines. Also, we find that levels in HKSL's hierarchy can learn to specialize in long- or short-term consequences of agent actions, thereby providing the downstream control policy with more informative representations. Finally, we determine that communication channels between hierarchy levels organize information based on both sides of the communication process, which improves sample efficiency.
@article{mcinroe2023hksl,
title = {Multi-Horizon Representations with Hierarchical Forward Models for Reinforcement Learning},
author = {Trevor McInroe and Lukas Schäfer and Stefano V. Albrecht},
journal = {Transactions on Machine Learning Research (TMLR)},
year = {2024}
}