What is a Model Hyperparameter?
A model hyperparameter is the parameter whose value is set before the model start training. They cannot be learned by fitting the model to the data.
Example:
In the above plot x-axis represents the number of epochs and y-axis represents the number of epochs. We can see after a certain point when epochs are more than then although the training accuracy increases but the validation and test accuracy starts decreasing. This is a case of overfitting. Here number of epochs is a hyperparameter and is set manually. Setting this number to a small value may cause underfitting and high value may cause overfitting.
Model hyperparameters in different models:
- Learning rate in gradient descent
- Number of iterations in gradient descent
- Number of layers in a Neural Network
- Number of neurons per layer in a Neural Network
- Number of clusters(k) in k means clustering
Difference Between Model Parameters VS HyperParameters
The two most confusing terms in Machine Learning are Model Parameters and Hyperparameters. In this post, we will try to understand what these terms mean and how they are different from each other.