Difference between Model Parameter and Hyperparameter in ML:

I thought to write about it because once a moderator asked me in interview. I think its very typical and tricky question

In a machine learning model, there are 2 types of parameters:

1. Model Parameters: These are the parameters in the model that must be determined using the training data set. These are the fitted parameters. For example: In linear regression model ,when we calculate best fit line we get slope and intercept of the model. So that m and c are the parameters.

Linear Regression

In the above plot, x is the independent variable, and y is the target variable. The objective is to fit a regression line to the data. This line(the model) is then used to predict the y-value for independent variables. Here, m is the slope and c is the intercept of the line. These two parameters(m and c) are estimated by fitting a straight line to the data by minimizing the RMSE(root mean squared error). Hence, these parameters are called the model parameters.

1. Hyperparameters: A model hyperparameter is the parameter whose value is set before the model start training. They cannot be learned by just fitting the model to the data. These are adjustable parameters that must be tuned in order to obtain a model with optimal performance.
2. Some examples of hyperparameters:

• Number of leaves or depth of a tree
• Alpha in regularization
• Learning rate (in many models)
• Number of hidden layers in a deep neural network
• Number of clusters in a k-means clustering

Thanks

Neha