Contents

Portfolio Management and Trading Optimization

This project aims to study the application of Reinforcement Learning techniques to portfolio management and trading optimization.

Portfolio Management (also called asset management) is the management of a set of assets whose main goal is to meet the requirements provided by the investors. Although this field has been studied since ten years, most of the research (both in machine learning and statistics) is focused on the provision of means for the analysis of the market, estimation of financial trends and monitoring of investments.

On the other hand, many characteristics of the problem of portfolio management (e.g., the final goal can be achieved through a sequence of actions, the inverstor objective is to maximize the global benefit and not only the instant one) can be effectively dealt with by Reinforcement Learning techniques as proved by recenent works on trading optimization in the NASDAQ market.

The main goal of this research project is to analyze the characteristics of the portfolio management problems and understand when Reinforcement Learning techniques can be applied to directly manage the composition of the portfolio
and improve the benefits for the investors. In particular, we will apply traditional RL algorithms and function approximators for the management of a single asset and we will exploit hierarchical decomposition techniques for the
management of a set of heterogeneous assets.

Function Approximation in Reinforcement Learning

This project aims to analyze the effects of function approximation techniques on Reinforcement Learning algorithms from a theoretical and experimental point of view.

Reinforcement Learning algorithms suffer from the curse of dimensionality and it is often unfeasible to deal with problems characterized by huge state and action spaces. Therefore, function approximation techniques (e.g., state aggregation, neural networks and so forth) are usually applied to learn an accurate approximation of the (action) value function with few parameters. Although function approximation has been widely applied to RL problems, many theoretical and practical issues are still open.

This research project aims at the analysis of theoretical issues such as convergence and optimality and at the development of novel algorithms that incrementally adapt their structure according to the characteristics of the problem at hand.

Intrinisically Motivated Reinforcement Learning

This project aims to design algorithms that mimic the development process of human being and animal puppies.

The main hypothesis of Reinforcement Learning is that a reinforcement signal is provided to the agent by a critic in the environment. On the other hand, human beings often learn without a direct and explicit reinforcement from the surrounding environment but they act guided by an intrinsic motivation at reaching one goal. This mechanism has been proven to be at the basis of many development processes in human beings and animal puppies. Recently, the Intrinsically Motivated Reinforcement Learning (IMRL) has been proposed as a framework to explain some of the development mechanisms that lead humans to develop skills and abilities through a direct interaction with the envirionment and guided by an intrinsic motivation.

This research project has many facets. The main goal is to deepen the biological and psychological plausibility of the IMRL framework to explain human development. At the same time, it could represent a fundamental step towards the definition of techniques to provide robots with the capability to improve their abilities and to develop new skills that can be used in many different tasks. Finally, this research project is strictly related to the project of transfer learning that aims at enabling artificial agents to reuse their knowledge in analogous situations.