Understanding Reinforcement Learning with Neural Networks Part 2: Why Backpropagation Is Not Enough

Short summary

Reinforcement learning differs from supervised learning because ideal outputs aren't known in advance. Backpropagation requires comparing outputs to known targets, so RL uses policy gradients to estimate derivatives from guesses. This explains why standard neural network training fails for RL problems.

• •Backpropagation requires known ideal outputs to calculate error gradients
• •In RL, ideal outputs are unknown until after actions are taken
• •Policy gradients solve this by estimating derivatives without labels

Generated with AI, which can make mistakes.