DefinePK

Abstract

Using a large number of variables in regression modeling might lead to a serious collinearity problem. Ordinary least square produces a very large bias and mean squared error of the parameter estimates and prediction. Several methods have been proposed to overcome collinearity problem such as a ridge, principal component, and partial least square regressions. Ridge regression is a method commonly used. Ridge regression method can be modeled using several ways such as classical, Bayesian, and machine learning. Our study has evaluated those methods to find the best method that can be used to overcome the collinearity problem. Apparently we observed the ridge regression by means machine learning method is the most powerful method. It has a minimum bias and mean squared error of the parameter estimates and minimum residual. This method provides an algorithm called gradient descent. Gradient descent can be used to estimate the optimum regression parameter by minimizing the cost function (reached the steepest descent). Thus the value of this function will update the previous parameters and it will give us the best model of the dataset.