Parameter and hyperparameter in Machine learning


What is Parameter and hyperparameter in Machine learning




Let’s find out what are parameters

Well, Parameters are nothing but the model parameters.  

What are the model parameters?  well model parameters are  something for example weights and biases of  neural network



Parameters in a machine learning model are the parameters Whose values are updated during training using some optimisation procedure()


Let’s talk about hyperparameters


 Hyperparameters are basically the parameters which are related to the training or learning of the algorithm

For example 

  1.  learning rate
  2.  batch size (all the data at once, mini  batch,  single data point  a time )
  3.  number of  epochs


Hyperparameters in a machine learning model are the parameters whose values are decided before training of the model begins. Hyperparameters are not affected (do not change) by training.  Rather, these affect the quality and speed of the training. So in a sense these parameters are over and beyond or above (hyper -) the process of training and hence called hyperparameters.

Hyperparameters can be related to model selection, e.g., model type , model architecture, or learning algorithm , e.g., learning rate , batch size, or number of epochs


There is no efficient algorithm to select optimal (best)  values of hyperparameters. So optimal values of hyper parameters are determined using a trial-and-error process by adjusting or adapting their values for a particular machine learning task. This process of determining values of hyperparameters is called hyperparameter tuning


Hyperparameter tuning is very similar to tuning the knobs of a guitar. Here one adjusts the tension of a string by adjusting the knobs until it gets the correct note.


Similarly In hyperparameter tuning we keep doing the trial with different hyperparameter values against several learning algorithms until we get the minimal error against a validation set 


Leave a Reply

Your email address will not be published. Required fields are marked *

Doubts? WhatsApp me !