# 数据集扩充

`horizontally flipping` `random crops` `color jittering`

# 预处理(normalization)

` 减去均值` `zscore` `白化(whittening)`

## 减去均值

` >> x -= np..mean(X, axis = 0) # zero-center`

## zscore

```>> X -= np.mean(X, axis = 0) # zero-center
>> X /= np.std(X, axis = 0) # normalize```

## 白化(whitening)

1. `PCA白化`
2. `ZCA白化`

### PCA whitening

pca白化是指对上面的pca的新坐标X’,每一维的特征做一个标准差归一化处理

### ZCA whitening

ZCA白化是在PCA白化的基础上，又进行处理的一个操作。具体的实现是把上面PCA白化的结果，又变换到原来坐标系下的坐标:

## 权重的初始化

Small Random Numbers

`>> 0.01 * N(0,1)    #N(0,1)表示均值为0的标准高斯分布`

Calibrating the Variances

`方差(variance)为2/n`

`>> w = np.random.randn(n) * sqrt(2.0/n) # current recommendation`

# Training

## Filter size

it is important to employ a small filter (e.g., `3*3`) and small strides (e.g., 1) with zeros-padding, which not only reduces the number of parameters, but improves the accuracy rates of the whole deep network. Meanwhile, a special case mentioned above, i.e., `3*3` filters with stride 1, could preserve the spatial size of images/feature maps. For the pooling layers, the common used pooling size is of `2*2`.

## Activation Functions

• `sigmoid`很少用,不推荐(kill gradients, not zero-centered)
• `tanh``sigmoid`要好(is zero-centered)
• `ReLU`系列: `ReLU`,`PReLU`,`Leaky ReLU`,`RReLU`,中推荐使用`PReLU` and `RReLU`

l2正则一般情况下优于l1正则

0.5的概率值是典型的做法

## 数据倾斜

### sampling techniques

1. duplicating instances(maybe special crops processing) from the minority classes until a balanced distribution is reached (oversampling) 2. removing instances from over-represented classes (undersampling) 3. it is suggested that a combination is the best solution for extremely imbalanced distributions 4. generating new data in minority classes based on the current data

### cost sensitive techniques

a higher penalty can be given to the network when it misclassifies the minority classes during training

### One-class learning

1. only provides training data from a single class(每一类去训练,不停的fine-tuning)
2. firstly fine-tune on the classes which have a large number of training samples (images/crops), and secondly, continue to fine-tune but on the classes with limited number samples

89 篇文章41 人订阅

0 条评论

## 相关文章

### 【干货】苹果 AI 负责人 Russ Salakhutdinov 最新演讲：深度生成模型定量评估（56 PPT）

【新智元导读】Russ Salakhutdinov 是苹果 AI 研发负责人，CMU 教授，也是著名的机器学习领域大牛。本文是 Russ 有关评估深度生成模型的...

3586

### 数据挖掘常用模型构建示例（R语言版）

Linear Regression library(MASS) lm_fit = lm(medv~poly(rm,2)+crim,data = Boston) ...

3085

3446

1344

### Batch Normalization论文翻译——中英文对照

Batch Normalization: Accelerating Deep Network Training by Reducing Internal Cov...

2121

5393

56411

33410

1783

### 统计学习导论 Chapter3--Linear Regression

Book: An Introduction to Statistical Learning with Applications in R http:...

2277