# 机器学习基本概念-3

## 从线性回归说起

Parameters(weights) are values that control the behavior of the system and determine how each feature affects the prediction.

## Regularization

Regularization prevents overfitting by adding penalty for complexity.

Equivalent to imposing a preference over the set of functions that a learner can obtain as a solution.o

cost function = error on the training set + regularization.

There are many other ways of expressing preferences for different solutions, both implicitly and explicitly. Together, these different approaches are known as regularization.

Instead we must choose a form of regularization that is well-suited to the particular task we want to solve.

## Function estimation

Prediction function f can be parametrized by a parameter vector θ.

estimating θ from D.

estimation的质量可以使用BiasVariance来衡量,而不是拿真实的parameter和学习到的parameter或者函数之间来对比.

## Bias

where expectation is over all the train sets of size n sampled from the underlying distribution.

Unbiased等价于bias==0:

## Variance

Variance典型的随着training set的增大而减小.

Variance typically decreases as the size of the train set increases.

Biasvariance到底是什么作用呢? Bengio是这样说的,我翻译不好,原话贴上:

Bias measures the expected deviation from the true value of the function or parameter. Variance on the other hand, provides a measure of the deviation from the expected estimator value that any particular sampling of the data is likely to cause.

Increasing the capacity of a learner may also increase variance, although it has better chance to cover the true function.

BiasVariance都是model的estimation error的一部分,比如:

89 篇文章41 人订阅

0 条评论

## 相关文章

### Hinton向量学院推出神经ODE：超越ResNet 4大性能优势

【导读】Hinton创建的向量学院的研究者提出了一类新的神经网络模型，神经常微分方程（Neural ODE），将神经网络与常微分方程结合在一起，用ODE来做预测...

1363

2765

1101

### 深度离散哈希算法，可用于图像检索！

-免费加入AI技术专家社群>> 智能感知与计算研究中心李琦博士提出了一种深度离散哈希算法（discrete hashing algorithm），该算法认为学习...

6386

40611

3394

6397

3056

4225

1144