Breaking News: Grepper is joining You.com. Read the official announcement!

gradient descent

Innocent Iguana answered on February 13, 2023 Popularity 10/10 Helpfulness 2/10

answer gradient descent

related gradient descent

gradient descent

Comment

# Initial function
def func1(x):
  return x**2-4*x+1

# Gradient of function
def gradient_func1(x):
  return 2*x - 4

# Gradient Descent algorithm
import numpy as np

def gradient_descent(start, gradient, learn_rate, max_iter, tol=0.01):
  steps = [start] # history tracking
  x = start

  for _ in range(max_iter):
    diff = learn_rate*gradient(x)
    if np.abs(diff)<tol:
      break    
    x = x - diff
    steps.append(x) # history tracing

  return steps, x

# Implementation
history, result = gradient_descent(9, gradient_func1, 0.1, 100)

# Final descent point
result

# Trace of descent
history

Popularity 10/10 Helpfulness 2/10 Language python

Source: Grepper

Tags: gradient-descent python

Link to this answer
Share Copy Link

Contributed on Apr 27 2023

Innocent Iguana

0 Answers Avg Quality 2/10

Closely Related Answers

gradient descent

Comment

A cost function estimates the error of a model.
Gradient descent is a technique that uses derivative of the cost function
to change the parameter values (weights / co-efficients) to minimize the 
cost or error.
Change the current weights by delta and take a step. The direction and size 
of step is calculated by the gradient (slope) of cost function at the 
current position by some specified learning rate. The gradient vector 
contains the slopes for all weight vectors / co-efficients. This gradient 
vector is used to update all existing weights.
If slope is negative, we are stepping downhill and will reach a 
minimum position when the slope becomes 0 / algorithm converges.
If slope is positive, we are stepping uphill and will reach a 
maximum position when the slope becomes 0 / algorithm converges.

Process of applying gradient descent:
1. Initialize parameters randomly
2. Calculate cost for training set with a cost function
3. Calculate gradient of the cost function (partial derivative for all dataset)
4. Update weights with new values
5. Repeat from step 2 until cost is small enough

- Normal gradient descent : 
	- One sample at a time
    - One run on all samples = 1 epoch
- Mini-batch gradient descent : 
	- few samples at a time
    - One run on one batch = 1 epoch

Popularity 10/10 Helpfulness 2/10 Language whatever

Source: Grepper

Tags: gradient-descent grad

Link to this answer
Share Copy Link

Contributed on Feb 13 2023

Innocent Iguana

0 Answers Avg Quality 2/10

gradient descent

Comment

- A method to find the minimum or maximum of a function by moving along the 
	slope by a positive or negative direction according to the sign of the 
    derivative
- Process / workflow:
    - goal: keep descending until the loss is minimized. Stop the process 
    	when the loss start increasing.
    - randomly take a number and calculate slope parameter
    - move the slope to a positive direction by adding a small number
    - move the slope to a negative direction by subtracting a small number
    - How large the number should be added is determined by a parameter called l
    	earning rate
        - too small learning rate = hardly any improvement in each iteration
        - too large learning rate = sometimes we will miss the minimum
    - calculate equation after each move to see which iteration provides the least loss 
    	(the minimum loss, global extrema)

Popularity 10/10 Helpfulness 1/10 Language whatever

Source: Grepper

Tags: gradient-descent grad

Link to this answer
Share Copy Link

Contributed on Jun 01 2023

Innocent Iguana

0 Answers Avg Quality 2/10

gradient descent

Contents

More Related Answers

gradient descent

Closely Related Answers

gradient descent

gradient descent

Grepper

Documentation

Social

Legal

Contact

Oops, You will need to install Grepper and log-in to perform this action.