Q-Learning is an off-policy temporal difference method that uses the update:
where is a learning rate, and the choice of action at each step is derived from a different behavior policy .
In SARSA and Q-Learning, the policy is updated by taking the maximum of the action values at each state (see this equation). It can be shown that these updates are contraction mappings; the action values will eventually converge, assuming that every state-action pair is visited an infinite number of times.