Course Content
derivatives and gradient
0/5
DEMO : gradients – using excel – notebook
00:31
Gradient descent
00:00
build linear regression using gradient descent
00:00
types of gradient descent
00:00
DEMO : build linear regression using Stochastic Gradient Descent
00:00
Stochastic models in sklearn
In scikit-learn, stochastic models refer to a class of algorithms that involve randomness in their training process. These models are typically used for large-scale datasets when it is computationally expensive or infeasible to process the entire dataset at once. Instead, they perform updates on a subset of the data or use random sampling techniques.
0/3
SGDClassifier
00:00
SGDRegressor
00:00
Mini-Batch K-Means
00:00
Machine Learning – Part 4 – Gradient Based Models (Pre-Cursor To Deep Learning)
About Lesson

  1. Cost Function: The goal of linear regression is to find the parameter values that minimize the difference between the predicted values and the actual target values in the training data. This is achieved by defining a cost function, typically the mean squared error (MSE) or the sum of squared errors (SSE):

    J(╬©) = (1/2m) * ╬ú(yßÁó – h╬©(xßÁó))┬▓

    where J(╬©) is the cost function, m is the number of training examples, yßÁó is the actual target value for the i-th example, h╬©(xßÁó) is the predicted value given the parameters ╬© and the input features xßÁó.

  2. Gradient Descent: Gradient descent is an iterative optimization algorithm that aims to find the minimum of the cost function by updating the parameter values in the direction of the steepest descent. The update rule for each parameter is:

    ╬©Ô▒╝ := ╬©Ô▒╝ – ╬▒ * (ÔêéJ(╬©)/Ôêé╬©Ô▒╝)

    where ╬©Ô▒╝ is the j-th parameter, ╬▒ is the learning rate (step size), and (ÔêéJ(╬©)/Ôêé╬©Ô▒╝) is the partial derivative of the cost function with respect to ╬©Ô▒╝.

  3. Gradient Descent Algorithm: The gradient descent algorithm for linear regression can be summarized as follows:

    • Initialize the parameter values ╬©Ô▒╝ randomly or with zeros.
    • Repeat until convergence:
      • Calculate the predicted values h╬©(xßÁó) for all training examples.
      • Update each parameter ╬©Ô▒╝ using the gradient descent update rule.
    • Once convergence is reached (or after a fixed number of iterations), the estimated parameter values ╬© provide the fitted linear regression model.
Join the conversation
Previous
Next