# 统计学习导论 Chapter5 -- Resampling Methods

Book: An Introduction to Statistical Learning with Applications in R http://www-bcf.usc.edu/~gareth/ISL/

5.1 Cross-Validation 首先来回顾一下 test error rate 和 training error rate，一个算法只有 test error rate ，其在实际应用中效果才可能比较好。如果有一个 designated test set，那么 test error rate 很容易被计算。 但是通常没有这个测试数据集。 training error rate 可以很容易在训练数据集中计算得到。但是 the training error rate often is quite different from the test error rate, and in particular the former can dramatically underestimate the latter.

5.1.1 The Validation Set Approach 我们将手中的样本分为两个部分：a training set and a validation set or hold-out set，我们首先在 训练数据集上拟合模型，然后再 validation set 测试模型 得到一个 validation set error rate，这个 validation set error rate 是 test error rate 的一个很好的估计

validation set approach 概念简单，也很容易实现。但是它有两个可能的缺点： 1）the validation estimate of the test error rate can be highly variable, depending on precisely which observations are included in the training set and which observations are included in the validation set 2）the validation set error rate may tend to overestimate the test error rate for the model fit on the entire data set

cross-validation, a refinement of the validation set approach that addresses these two issues cross-validation 是对上面两个问题的解决方案

5.1.2 Leave-One-Out Cross-Validation Leave-one-out cross-validation (LOOCV) 和上面的 validation set approach 很紧密，但是可以解决其缺陷 和 validation set approach 类似， LOOCV 也是将数据集分为两个部分，但是我们每次只选择一个样本作为 validation set，剩下的样本作为 training set。统计学习方法在 这 n-1 个样本的 training set 进行模型拟合，在 validation set 进行测试，得到误差 MSE1 。如果数据集中每个样本都当过一次 validation set，我们就有 n 个 MSE。The LOOCV estimate for the test MSE is the average of these n test error estimates

LOOCV 相对于 validation set approach 的优点： 1） it has far less bias 2） performing LOOCV multiple times will always yield the same results: there is no randomness in the training/validation set splits

LOOCV 的缺点就是计算量比较大

With least squares linear or polynomial regression, an amazing shortcut makes the cost of LOOCV the same as that of a single model fit!

5.1.3 k-Fold Cross-Validation An alternative to LOOCV is k-fold CV. This approach involves randomly dividing the set of observations into k groups, or folds, of approximately equal size. The first fold is treated as a validation set, and the method is fit on the remaining k − 1 folds

In practice, one typically performs k-fold CV using k = 5 or k = 10. What is the advantage of using k = 5 or k = 10 rather than k = n? The most obvious advantage is computational

5.1.4 Bias-Variance Trade-Off for k-Fold Cross-Validation 如果暂时不考虑计算量的问题 a less obvious but potentially more important advantage of k-fold CV is that it often gives more accurate estimates of the test error rate than does LOOCV. This has to do with a bias-variance trade-off.

To summarize, there is a bias-variance trade-off associated with the choice of k in k-fold cross-validation. Typically, given these considerations, one performs k-fold cross-validation using k = 5 or k = 10, as these values have been shown empirically to yield test error rate estimates that suffer neither from excessively high bias nor from very high variance.

5.2 The Bootstrap we instead obtain distinct data sets by repeatedly sample observations from the original data set.

0 条评论

## 相关文章

2779

5114

### 深度学习目标检测指南：如何过滤不感兴趣的分类及添加新分类？

AI 科技大本营按：本文编译自 Adrian Rosebrock 发表在 PyImageSearch 上的一篇博文。该博文缘起于一位网友向原作者请教的两个关于目...

1352

### 深度学习之图像的数据增强

在图像的深度学习中，为了丰富图像训练集，更好的提取图像特征，泛化模型（防止模型过拟合），一般都会对数据图像进行数据增强， 数据增强，常用的方式，就是旋转图...

9237

7488

### 深度学习与TensorFlow:FCN论文翻译(二)

Each layer of data in a convnet is a three-dimensional array of size h × w × d, ...

2252

4176

55410

1213

### 目标检测--RON: Reverse Connection with Objectness Prior Networks for Object Detection

RON: Reverse Connection with Objectness Prior Networks for Object Detection CV...

4647