# cs231n之KNN、SVM

0.说在前面1.KNN2.SVM3.作者的话

## 1.KNN

np.flatnonzero

```idxs = np.flatnonzero(y_train == y)
```

In:

```import numpy as np
x = np.arange(-3,5)
```

Out:

```array([-3, -2, -1,  0,  1,  2,  3,  4])
```

In:

```np.flatnonzero(x)
```

Out:

```array([0, 1, 2, 4, 5, 6, 7], dtype=int64)
```

In:

```np.flatnonzero(x==-3)
```

Out:

```array([0], dtype=int64)
```

np.random.choice

```numpy.random.choice(a, size=None, replace=True, p=None)
```

a

size

replace

p

```np.random.choice(5,3) # 等价于np.random.randint(0,5,3)
```

p参数：

```np.random.choice(5,3,p=[0,0,0.1,0.6,0.3])
```

replace参数：

In:

```a = ['bird','meh','sad','d','123']
np.random.choice(a,5) # replace默认为True
np.random.choice(a,5,replace=False)
```

Out:

```array(['d', 'd', 'meh', '123', 'sad'], dtype='<U4')
array(['d', 'bird', 'meh', '123', 'sad'], dtype='<U4')
```

reshape中-1

In:

```x = np.arange(-3,5)
x
```

Out:

```array([-3, -2, -1,  0,  1,  2,  3,  4])
```

-1可以自动计算出数组的列数或行数

In:

```x.reshape(-1,1)
```

Out:

```array([[-3],
[-2],
[-1],
[ 0],
[ 1],
[ 2],
[ 3],
[ 4]])
```

In

```x.reshape(1,-1)
```

out:

```array([[-3, -2, -1,  0,  1,  2,  3,  4]])
```

```(5000, 3072) (500, 3072)
```

```# 传进来x_test shape(500,3072)
# self.X_train shape(5000,3072)
def compute_distances_two_loops(self, X):
# 500
num_test = X.shape[0]
# 5000
num_train = self.X_train.shape[0]
# 生成一个shape(500,5000)的全为0矩阵
dists = np.zeros((num_test, num_train))
# 两重for循环
for i in range(num_test):
for j in range(num_train):
# 利用欧式距离计算
dists[i，j]=np.sqrt(np.sum(np.square(X[i，:]-self.X_train[j，:])))
return dists
```

```x = np.array([5,3,0,-1,9])
np.argsort(x)
```

```array([3, 2, 1, 0, 4], dtype=int64)
```

```# y_train shape(5000,)
def predict_labels(self, dists, k=1):
num_test = dists.shape[0]
y_pred = np.zeros(num_test)
# 上面两层循环的得出dists shape(500,5000)
# 下面通过循环来得出测试图片500张与训练图片5000张距离最近的前k个
for i in range(num_test):
closest_y = []
# 目的是得出测试图片500张与训练图片5000张距离最近的前k个
closest_y = self.y_train[np.argsort(dists[i，：])[:k]]
# 投票得出预测值
y_pred[i] = np.argmax(np.bincount(closest_y))
return y_pred
```

```x = np.array([0,1,2,2,1,1])
```

```np.bincount(x)
```

```array([1, 3, 2], dtype=int64)
```

0出现1次，1出现3次，2出现2次，那么再通过argmax可以求出最多次数的label

```np.argmax(np.bincount(x))
```

```1
```

```def compute_distances_one_loop(self, X):
# 500
num_test = X.shape[0]
# 5000
num_train = self.X_train.shape[0]
# (500,5000)
dists = np.zeros((num_test, num_train))
for i in range(num_test):
dists[i, :] = np.sqrt(np.sum(np.square(self.X_train - X[i, :]), axis=1))
return dists
```

X_train为(5000,3072)，而X为(500,3072)，对比维度

```                维度:  0     1
self.X_train          5000  3072
X                     500   3072
```

```                维度:  0     1
self.X_train          5000  3072
X[i,:]                1     3072
```

```t = np.sum([[0,1,2],[2,1,3]],axis=1)
t.shape
```

```(2,)
```

```def compute_distances_no_loops(self, X):
num_test = X.shape[0]
num_train = self.X_train.shape[0]
dists = np.zeros((num_test, num_train))
dists = np.sqrt(np.sum(X**2, axis=1, keepdims=True) + (-2 * np.dot(X, self.X_train.T)) + np.sum(self.X_train**2, axis=1))
return dists
```

```np.sum(a**2,axis=1)
```

```array([ 5, 25], dtype=int32)
```

```np.sum(a**2,axis=1,keepdims=True)
```

```array([[ 5],
[25]], dtype=int32)
```

```                维度:  0     1
x^2-2xy              500  5000
self.X_train^2       5000   1
```

```x = np.arange(6)
np.array_split(x,3,axis=0)
```

```[array([0, 1]), array([2, 3]), array([4, 5])]
```

```X_train_folds = np.array_split(X_train, num_folds, axis=0) # list
y_train_folds = np.array_split(y_train, num_folds, axis=0) # list
```

```# num_folds = 5
for i in range(num_folds):
# train / validation split (80% 20%)
X_train_batch = np.concatenate(X_train_folds[1:num_folds])
y_train_batch = np.concatenate(y_train_folds[1:num_folds])
X_valid_batch = X_train_folds[0]
y_valid_batch = y_train_folds[0]
# swap data (for next iteration)
if i < num_folds - 1:
tmp = X_train_folds[0]
X_train_folds[0] = X_train_folds[i+1]
X_train_folds[i+1] = tmp
tmp = y_train_folds[0]
y_train_folds[0] = y_train_folds[i+1]
y_train_folds[i+1] = tmp
# train model
model = KNearestNeighbor()
model.train(X_train_batch, y_train_batch)
dists = model.compute_distances_no_loops(X_valid_batch)
# compute accuracy for each k
for k in k_choices:
y_valid_pred = model.predict_labels(dists, k=k)
# compute validation accuracy
num_correct = np.sum(y_valid_pred == y_valid_batch)
accuracy = float(num_correct) / y_valid_batch.shape[0]
# accumulate accuracy into dictionary
if i == 0:
k_to_accuracies[k] = []
k_to_accuracies[k].append(accuracy)
```

## 2.SVM

```Training data shape:  (49000, 3072)
Validation data shape:  (1000, 3072)
Test data shape:  (1000, 3072)
dev data shape:  (500, 3072)
```

`np.hstack`水平扩展

```X_train = np.hstack([X_train, np.ones((X_train.shape[0], 1))])
```

svm native

X是矩阵，W是权重(Wi表示每一类别权重)

X表示如下：

W表示如下：

S表示如下：

S中每一行对应

（6）式中wjxi表示第j个类别的得分！

Li表示(3)式中的每一行loss，Li的梯度写成如7所示的列向量，将每一行的loss分成每一个元素的loss，即公式(7)。

```def svm_loss_naive(W, X, y, reg):
# C
num_classes = W.shape[1]
# N
num_train = X.shape[0]
loss = 0.0
for i in range(num_train):
# Li 每一行，对应(3)式
scores = X[i].dot(W)
# scores[y[i]] WyjXi 真正lable得分
correct_class_score = scores[y[i]]
# 每一列
for j in range(num_classes):
# j=y[i]不能求和，跳过
if j == y[i]:
continue
# Lij 对应(6)式中的每一个Lij
margin = scores[j] - correct_class_score + 1 # note delta = 1
# score>0
if margin > 0:
# 对应(6)式求和
loss += margin
# 对应(10)式
dW[:, y[i]] += -X[i, :]
# 对应(9)式
dW[:, j] += X[i, :]
```

0 条评论

• ### SVM梯度求导及实现

昨晚看了一部电影，叫做我是马布里，非常正能量，推荐给各位，看完这部电影的总结话是：

• ### 再谈强大的Numpy

今天发现cs231n还差一个features.py未更新，特更，并且更新中间穿插的numpy使用！

• ### Softmax及两层神经网络

0.说在前面1.Softmax向量化1.1 Softmax梯度推导1.2 Softmax向量化实现2.两层神经网络2.1 反向传播推导2.2 两层神经网络实现3...

• ### 斯坦福CS231n项目实战（三）：Softmax线性分类

版权声明：本文为博主原创文章，未经博主允许不得转载。 https://blog.csdn.n...

• ### Occupancy Networks：基于学习函数空间的三维重建表示方法

随着深度神经网络的到来，基于学习的三维重建方法逐渐变得流行。但是和图像不同的是，在3D中没有规范的表示，既能高效地进行计算，又能有效地存储，同时还能表示任意拓扑...

• ### 【解决方案】TSINGSEE青犀视频云边端架构RTSP视频边缘计算平台EasyNVR搭建“云上透明厨房”远程监控平台方案

餐饮行业对操作和环境要求都比较高。很多直营的餐饮点由于店面较多，人员层次不齐，时长发现一些不和规范的操作，导致响应口碑下载，这样严重影响餐饮企业的品牌口碑。

• ### Laravel 5.5 在浏览器中渲染 Mailable 类型

Laravel 框架中提供了很方便快捷的面向对象风格的电子邮件相关功能，可以通过 Markdown 语法、Blade 模板引擎来制作邮件模板，然后通过扩展 Ma...

• ### 一文详解JDK13新特性

核心库/ java.nio中添加了FileSystems.newFileSystem（Path，Map <String，？>）方法

• ### 一篇文章看清楚JDK13的特性！

提高应用程序类 - 数据共享（AppCDS）的可用性。消除了用户进行试运行以创建每个应用程序的类列表的需要。