State-Action-Reward-State-Action (SARSA) is an on-policy temporal difference method, using the update:
where is the learning rate. The bracketed term is called the TD error and measures the consistency between the estimated action value and the estimate after taking a single step.
In SARSA and Q-Learning, the policy is updated by taking the maximum of the action values at each state (see this equation). It can be shown that these updates are contraction mappings; the action values will eventually converge, assuming that every state-action pair is visited an infinite number of times.