Reinforcement Learning with Qualitative Feedback

Reinforcement learning (RL) is an estabilshed paradigm for autonomous learning from interaction with an environment in order to achieve long-term goals. Informally speaking, the task of an RL agent is to successively react to changes of the environment, by properly chossing actions, in order to maximize its accumulated reward over time. Conventional RL methods are confined to deal with numerical rewards. However, in many applications, only qualitative reward signals are readily available, Moreover, a restriction to numerical reward functions also hampers the exploitation of other conceivable sources of feedback, such as external advice.
Our goal in this project is to generalize the standard RL framework so as to allow more general types of feedback, notably non-numerical rewards and qualitative advice. Building on novel methods, such as ranking functions that allow for sorting such models. While the focus of the project is on the development of theoretical and methodological foundations of a "preference-based reinforcement learning", we also envision two case studies putting our ideas into practice, one in the field of game playing and another one in a medical context.

Publications