Rectified Linear Unit

The rectified linear unit is a standard activation function; it returns the input when it is positive and zero otherwise:

$$a[z]=\text{ReLU}[z]=\{\begin{array}{ll}0& z<0\\ z& z\ge 0\end{array}=\text{max}(0,z)$$