2017年2月18日 Stochastic Gradient Descent of Linear Model 原

airxiechao

Stochastic Gradient Descent iterates through all samples to update parameters of model by gradient descent

``````import numpy as np
import ml_metrics as mtr
from collections import deque

X = data.data
y = data.target

class mySGD:

def derivative_of_cost(self, x, y, w):
return 2*x*(np.dot(x,w)-y)

def gradient_descent(self, x, y, w, eta):
return w - eta*self.derivative_of_cost(x, y, w)

def fit(self, X, y, eta):
w = np.zeros(X.shape[1])

vq = deque()
wq = deque()
vq_min = np.inf
num_tolerance = 20

while True:
for n in range(X.shape[1]):
w = self.gradient_descent(X[n], y[n], w, eta)

wq.append(w)
if len(wq) > num_tolerance:
wq.popleft()

v = mtr.mse(y, np.dot(X,w))
vq.append(v)
if len(vq) > num_tolerance:
vq.popleft()

if len(vq) >= num_tolerance and min(vq) > vq_min:
w = wq[np.argmin(vq)]
break
else:
vq_min = min(vq)

self.w = w

def score(self, X, y):
return mtr.mse(np.dot(X,self.w), y)

sgd = mySGD()
sgd.fit(X,y,.0000001)
print 'score:', sgd.score(X,y)
#score: 82.2466169262``````

airxiechao

Optimization Methods Until now, you’ve always used Gradient Descent to update the parameters and minimize the cost. In this notebook, you will learn more advanced optimization......

bxg1065283526
2018/05/06
0
0
Keras 深度学习框架介绍----一起来慢慢走进deep learning

Introduce Keras是一个高级API，用Python编写，能够在TensorFlow、Theano或CNTK上运行。Keras提供了一个简单和模块化的API来创建和训练神经网络，隐藏了大部分复杂的细节。 How to install k...

qq_15642411
2018/04/20
0
0
【机器学习】回归分析、过拟合、分类

2017/11/27
0
0
deep learning 作業 2.2

Optimization Methods Until now, you've always used Gradient Descent to update the parameters and minimize the cost. In this notebook, you will learn more advanced optimization m......

zzbb
2018/05/20
113
0
Gentle Introduction to the Adam Optimization Algorithm for Deep Learning

The choice of optimization algorithm for your deep learning model can mean the difference between good results in minutes, hours, and days. The Adam optimization algorithm is an......

2018/12/04
24
0

mysql-connector-java升级到8.0后保存时间到数据库出现了时差

ValSong
18分钟前
3
0
Spring Boot 如何部署到 Linux 中的服务

honeymoose
20分钟前
2
0
Spring Boot 2 实战：使用 Spring Boot Admin 监控你的应用

1. 前言 生产上对 Web 应用 的监控是十分必要的。我们可以近乎实时来对应用的健康、性能等其他指标进行监控来及时应对一些突发情况。避免一些故障的发生。对于 Spring Boot 应用来说我们可以...

6
0
ZetCode 教程翻译计划正式启动 | ApacheCN

ApacheCN_飞龙

4
0
CSS定位

CSS定位 relative相对定位 absolute绝对定位 fixed和sticky及zIndex relative相对定位 position特性：css position属性用于指定一个元素在文档中的定位方式。top、right、bottom、left属性则...

studywin

7
0