Negative log-likelihood multi-class

We can extend the idea of Negative log-likelihood directly to multi-class classification with $K$ classes, where the training labels is represented with the one-hot vector $y=[y_1,\dots ,y_k{]}^T$, where $y_k=1$ if the example is of class $k$.

Assume that our network uses Softmax as the activation function in the last layer, so that the output is $a=[a_1,\dots ,a_k{]}^T$, which represents a probability distribution over $K$ possible classes. Then, the probability that our network predicts the correct class for this example is ${\prod }_{k=1}^K{a}_k^{y_k}$ and the log of the probability that it is correct is ${\sum }_{k=1}^K{y}_k\log a_k$, so

$${\mathcal{L}}_{\text{nllm}}(\text{guess, actual})=-\sum _{k=1}^K{\text{actual}}_k,\log ({\text{guess}}_k)$$

We’ll call this NLLM for negative log likelihood multiclass.