D-DL4CV-Lec05d-Convex_Function

Convex Function

Introduction

Intuition: A convex function is a bowl shape function. When we connect any two points on a convex function, for any point $(x,y,z)$ on the line, we can find $(x,y,z^{\prime })$ on the convex function where $z^{\prime }\le z$

We can extend this intuition to higher dimensions

Formal Definition: A function $f:X\in {\mathbb{R}}^N\to \mathbb{R}$ is convex if for all $x_1,x_2\in X$, $t\in [0,1]$,

$$f(t{x}_1+(1-t)x_2)\le tf(x_1)+(1-t)f(x_2)$$

Why is it important

If the loss function is convex, then we can guarantee that after optimization, the minimum we achieve must be the global minimum of the function