A helpful website that explains how policy-based deep reinforcement learning works:
https://medium.com/deep-math-machine-learning-ai/ch-13-deep-reinforcement-learning-deep-q-learning-and-policy-gradients-towards-agi-a2a0b611617e

University of Michigan - Ann Arbor

Policy-based methods learn the policy function directly, without calculating a value function for each action. These methods use a learning signal derived from sampling
instantiations of policy parameters and the set of policies is developed
towards policies that achieve better returns.

An example of a policy-based algorithm is Policy Gradient Methods.

Policy-Based Methods for Deep Reinforcement Learning

In policy gradients , we directly learn the policy function π which gives the probability distribution of actions.
$\pi(s,a;\theta)\rightarrow [0,1]$ Probability of taking action a given state s with parameters $\theta$.

Neural networks can be used to find the policy function.  The network takes state as an input and produces the probability distribution of actions.

Process in simple terms:
 - Takes in a state and gets the probability of each action
 - Selects the most probable action and observe next state & reward
 - Repeats until the end of the game and evaluates the total rewards
 - Updates the parameters in the network, based on the rewards, using backpropagation

In this way, the network allows the agent to play freely, but with every successive game, it provides better probabilities for actions that will lead the agent to a positive result.

Policy Gradient Methods for Deep Reinforcement Learning

In value-based methods, we calculate the Q values for all possible actions in the action space, but in real world , the number of actions can be a lot or can’t be imaginable. For example, the Robot walking problem has continuous and high dimensional action space. 

In policy based methods, we directly learn our policy function π without worrying about a value function, which means we can choose actions without calculate the Q(S,A) values.

Learn Before

Related