So on q learning I am assuming that the next action I will take will be optimal, I think that is what makes it off policy Also called off-policy temporal difference learning algorithm

Sarsa is on policy, that the difference

Actor network is network that learns probability of actions given state

My own questions that I am trying to find answers to

  1. I am tryung to understand the idea of generalization, like in ML the models generalize because we are learning a pattern in the data, like there is a relation between the inputs and the outputs, where is this in RL . Remeber the image in the bishop book of the sin wave