In this course, you will learn the basic concepts and algorithms of Reinforcement Learning. We will start with tabular methods, which use a table of values to keep track of the best actions to take in each situation. Then, we will learn how to adapt these algorithms to more complex tasks (with infinite possible states). Later, we will make a short introduction to neural networks and learn how to apply them to Reinforcement Learning tasks. Finally, we will learn some advanced algorithms: Deep SARSA, DQN, REINFORCE, and A2C.

We will use several Python libraries during the course. To simulate the tasks that our algorithms will face, we will use the Gym library. Moreover, during the first half of the course, we will use the NumPy numerical computation library to implement our algorithms. In the second half, we will switch to the PyTorch library to implement the advanced algorithms using neural networks.

• Be comfortable programming in Python
• Know basic linear algebra and calculus (matrices, vectors, determinants, derivatives, etc.)
• Know basic statistics and probability theory (mean, variance, normal distribution, etc.)

Part 1 (Tabular methods):

• Markov decision process (MDP)
• Dynamic programming
• Monte Carlo methods
• Temporal difference methods (SARSA, Q-Learning)
• N-step bootstrapping

Part 2 (Continuous state spaces):

• State aggregation
• Tile Coding

Part 3 (Deep Reinforcement Learning):

• Deep SARSA
• Deep Q-Learning
• REINFORCE
• Advantage Actor-Critic (A2C)