Loss Function

Definition

The loss function (or objective function, cost function) tells how good our current classifier is

Low loss = good classifier High loss = bad classifier

Negative Loss Function

The negative loss function (reward function) return high loss when the current classifier is good and vice versa

Mathematical Perspective

Give a dataset of example

$${\left\{\left(x_i,y_i\right)\right\}}_{i=1}^N$$

Loss for a single example is

$$L_i\left(f\left(x_i,W,b\right),y_i\right)$$

Loss for the dataset is average of per-example losses:

$$L=\frac{1}{N}\sum _i{L}_i\left(f\left(x_i,W,b\right),y_i\right)$$

---

Different Loss Function

Cross-Entropy Loss (Multinomial Logistic Regression)

D-DL4CV-Lec03ba-Cross-Entropy_Loss

Multiclass Support Vector Machine (SVM) Loss

D-DL4CV-Lec03bb-SVM_Loss