# 机器学习基本概念-3

## 从线性回归说起

Parameters(weights) are values that control the behavior of the system and determine how each feature affects the prediction.

## Regularization

Regularization prevents overfitting by adding penalty for complexity.

Equivalent to imposing a preference over the set of functions that a learner can obtain as a solution.o

cost function = error on the training set + regularization.

There are many other ways of expressing preferences for different solutions, both implicitly and explicitly. Together, these different approaches are known as regularization.

Instead we must choose a form of regularization that is well-suited to the particular task we want to solve.

## Function estimation

Prediction function f can be parametrized by a parameter vector θ.

estimating θ from D.

estimation的质量可以使用BiasVariance来衡量,而不是拿真实的parameter和学习到的parameter或者函数之间来对比.

## Bias

where expectation is over all the train sets of size n sampled from the underlying distribution.

Unbiased等价于bias==0:

## Variance

Variance典型的随着training set的增大而减小.

Variance typically decreases as the size of the train set increases.

Biasvariance到底是什么作用呢? Bengio是这样说的,我翻译不好,原话贴上:

Bias measures the expected deviation from the true value of the function or parameter. Variance on the other hand, provides a measure of the deviation from the expected estimator value that any particular sampling of the data is likely to cause.

Increasing the capacity of a learner may also increase variance, although it has better chance to cover the true function.

BiasVariance都是model的estimation error的一部分,比如:

89 篇文章41 人订阅

0 条评论

## 相关文章

3469

833

2522

### 开发 | Google 软件工程师解读：深度学习的activation function哪家强？

AI科技评论按：本文作者夏飞，清华大学计算机软件学士，卡内基梅隆大学人工智能硕士。现为谷歌软件工程师。本文首发于知乎，AI科技评论获授权转载。 ? TLDR (...

3794

1855

3427

### TensorFlow 深度学习笔记 卷积神经网络

Convolutional Networks deep dive into images and convolutional models Convnet Ba...

1928

2175

421

2393