-
Cost Function: The goal of linear regression is to find the parameter values that minimize the difference between the predicted values and the actual target values in the training data. This is achieved by defining a cost function, typically the mean squared error (MSE) or the sum of squared errors (SSE):
J(╬©) = (1/2m) * ╬ú(yßÁó – h╬©(xßÁó))┬▓
where J(╬©) is the cost function, m is the number of training examples, yßÁó is the actual target value for the i-th example, h╬©(xßÁó) is the predicted value given the parameters ╬© and the input features xßÁó.
-
Gradient Descent: Gradient descent is an iterative optimization algorithm that aims to find the minimum of the cost function by updating the parameter values in the direction of the steepest descent. The update rule for each parameter is:
╬©Ô▒╝ := ╬©Ô▒╝ – ╬▒ * (ÔêéJ(╬©)/Ôêé╬©Ô▒╝)
where ╬©Ô▒╝ is the j-th parameter, ╬▒ is the learning rate (step size), and (ÔêéJ(╬©)/Ôêé╬©Ô▒╝) is the partial derivative of the cost function with respect to ╬©Ô▒╝.
-
Gradient Descent Algorithm: The gradient descent algorithm for linear regression can be summarized as follows:
- Initialize the parameter values ╬©Ô▒╝ randomly or with zeros.
- Repeat until convergence:
- Calculate the predicted values h╬©(xßÁó) for all training examples.
- Update each parameter ╬©Ô▒╝ using the gradient descent update rule.
- Once convergence is reached (or after a fixed number of iterations), the estimated parameter values ╬© provide the fitted linear regression model.
About Lesson
Join the conversation