Introduction
In this lecture and the next lecture, we’ll fully introduce how to actually train a neural network from scratch in practice. This lecture covers
Content
Activation Function
This section we discuss the pros and cons of different activation functions
Data Preprocessing
This section discuss different ways in preprocessing input data
Weight Initialization
Bad weight initialization can cause hard optimization. This section introduces some evident way of initializing
Regularization
Aside from L1 and L2 regularization, there are still other way of regularizing In training process, regularization add randomness to the training process while in testing process, we average out randomness