TVP

在Python实现梯度下降优化算法

·获得使F(x)最小化的函数

·初始化一个值x，从该值开始下降就进行优化

·指定一个学习率，该学习率将确定要下降多少步，或收敛到最小值的速度有多快

·得到x的导数(下降)

·继续下降，这个值的导数乘以学习速率

·不断更新x的值

·检查您的停止状态，看是否要停止

·如果条件满足，停止。如果没有，则使用新的x值继续第4步，并保持重复算法

import numpy as np

import matplotlib.pyplot as plt

%matplotlib inline

creating the function and plotting it

function = lambda x: (x ** 3)-(3 *(x ** 2))+7

#Get 1000 evenly spaced numbers between -1 and 3 (arbitratil chosen to ensure steep curve)

x = np.linspace(-1,3,500)

#Plot the curve

plt.plot(x, function(x))

plt.show()

def deriv(x):

'''

Description: This function takes in a value of x and returns its derivative based on the

initial function we specified.

Arguments:

x - a numerical value of x

Returns:

x_deriv - a numerical value of the derivative of x

'''

x_deriv = 3* (x**2) - (6 * (x))

return x_deriv

def step(x_new, x_prev, precision, l_r):

'''

Description: This function takes in an initial or previous value for x, updates it based on

steps taken via the learning rate and outputs the most minimum value of x that reaches the precision satisfaction.

Arguments:

x_new - a starting value of x that will get updated based on the learning rate

x_prev - the previous value of x that is getting updated to the new one

precision - a precision that determines the stop of the stepwise descent

l_r - the learning rate (size of each descent step)

Output:

1. Prints out the latest new value of x which equates to the minimum we are looking for

2. Prints out the the number of x values which equates to the number of gradient descent steps

3. Plots a first graph of the function with the gradient descent path

4. Plots a second graph of the function with a zoomed in gradient descent path in the important area

'''

# create empty lists where the updated values of x and y wil be appended during each iteration

x_list, y_list = [x_new], [function(x_new)]

# keep looping until your desired precision

while abs(x_new - x_prev) > precision:

# change the value of x

x_prev = x_new

# get the derivation of the old value of x

d_x = - deriv(x_prev)

# get your new value of x by adding the previous, the multiplication of the derivative and the learning rate

x_new = x_prev + (l_r * d_x)

# append the new value of x to a list of all x-s for later visualization of path

x_list.append(x_new)

# append the new value of y to a list of all y-s for later visualization of path

y_list.append(function(x_new))

print ("Local minimum occurs at: "+ str(x_new))

print ("Number of steps: " + str(len(x_list)))

plt.subplot(1,2,2)

plt.scatter(x_list,y_list,c="g")

plt.plot(x_list,y_list,c="g")

plt.plot(x,function(x), c="r")

plt.show()

plt.subplot(1,2,1)

plt.scatter(x_list,y_list,c="g")

plt.plot(x_list,y_list,c="g")

plt.plot(x,function(x), c="r")

plt.xlim([1.0,2.1])

plt.title("Zoomed in Gradient descent to Key Area")

plt.show()

#Implement gradient descent (all the arguments are arbitrarily chosen)

step(0.5, 0, 0.001, 0.05)

• 发表于:
• 原文链接https://kuaibao.qq.com/s/20181016A1OU3K00?refer=cp_1026
• 腾讯「腾讯云开发者社区」是腾讯内容开放平台帐号（企鹅号）传播渠道之一，根据《腾讯内容开放平台服务协议》转载发布内容。
• 如有侵权，请联系 cloudcommunity@tencent.com 删除。

2024-04-12

2024-04-12

2024-04-12

2024-04-12

2024-04-12

2024-04-12

2024-04-12

2024-04-12

2024-04-12

2024-04-12

2024-04-12

2024-04-12

2024-04-12

2024-04-12

2024-04-12

2024-04-12

2024-04-12

2024-04-12

2024-04-12

2024-04-12