# Directly applying Bayesian ridge regression直接使用贝叶斯岭回归

In the Using ridge regression to overcome linear regression's shortfalls recipe, we discussed the connections between the constraints imposed by ridge regression from an optimization standpoint. We also discussed the Bayesian interpretation of priors on the coefficients, which attract the mass of the density towards the prior, which often has a mean of 0 .

So, now we'll look at how we can directly apply this interpretation though scikit-learn.

Ridge and lasso regression can both be understood through a Bayesian lens as opposed to an optimization lens. Only Bayesian ridge regression is implemented by scikit-learn, but in the How it works... section, we'll look at both cases.

First, as usual, let's create some regression data:首先，让我们生成回归数据集：

```from sklearn.datasets import make_regression
X, y = make_regression(1000, 10, n_informative=2, noise=20)```

How to do it...如何做的

We can just "throw" ridge regression at the problem with a few simple steps:在这个问题中，我们只需要丢给岭回归很少的步骤

```from sklearn.linear_model import BayesianRidge
br = BayesianRidge()```

The two sets of coefficients of interest are alpha_1 / alpha_2 and lambda_1 / lambda_2 .The alphas are the hyperparameters for the prior over the alpha parameter, and the lambda are the hyperparameters of the prior over the lambda parameter.

First, let's fit a model without any modification to the hyperparameters:

```br.fit(X, y)
br.coef_
array([-0.37073297,  0.16745965, -0.77672044, 29.24241894, -0.69319217,
0.64905847, 86.9454228 , -0.24738249, -1.63909699,  1.43038709])```

Now, if we modify the hyperparameters, notice the slight changes in the coefficients:

```br_alphas = BayesianRidge(alpha_1=10, lambda_1=10)
br_alphas.fit(X, y)
br_alphas.coef_
array([-0.36917484,  0.16682313, -0.77961059, 29.21596299, -0.69730227,
0.64425288, 86.86658136, -0.2477023 , -1.63266313,  1.42687844])```

How it works...如何工作的

For Bayesian ridge regression, we assume a prior over the errors and alpha. Both these priors are gamma distributions.The gamma distribution is a very flexible distribution. Here are some of the different shapes the gamma distribution can take given the different parameterization techniques for location and scale. 1e-06 is the default parameterization of BayesianRidge in scikit-learn:

As you can see, the coefficients are naturally shrunk towards 0 , especially with a very small location parameter.

There's more...扩展阅读

Like I mentioned earlier, there's also a Bayesian interpretation of lasso regression. Imagine we set priors over the coefficients; remember that they are random numbers themselves.

For lasso regression, we will choose a prior that naturally produces 0s, for example, the double exponential.

Notice the peak around 0. This will naturally lead to the zero coefficients in lasso regression.By tuning the hyperparameters, it's also possible to create 0 coefficients that more or less depend on the setup of the problem.

• ### Feature selection特征选择

This recipe along with the two following it will be centered around automatic fe...

• ### Poor man's grid search穷举网格搜索

In this recipe, we're going to introduce grid search with basic Python, though w...

• ### Using truncated SVD to reduce dimensionality使用截断奇异值进行降维

Truncated Singular Value Decomposition (SVD) is a matrix factorization technique...

• ### 周练19.11.03/10

它表示一个迷宫，其中的1表示墙壁，0表示可以走的路，只能横着走或竖着走，不能斜着走，要求编程序找出从左上角到右下角的最短路线。

• ### ZOJ 3705 Applications

Recently, the ACM/ICPC team of Marjar University decided to choose some new memb...

• ### US oil prices turn negative as demand dries up

The price of US oil has turned negative for the first time in history.

• ### hdu 2473 Junk-Mail Filter (并查集之点的删除)

Junk-Mail Filter Time Limit: 15000/8000 MS (Java/Others)    Memory Limit: 32768/...

• ### HDUOJ----（1031）Design T-Shirt

Design T-Shirt Time Limit: 2000/1000 MS (Java/Others)    Memory Limit: 65536/327...

• ### Playrix Codescapes Cup (Codeforces Round #413, rated, Div. 1 + Div. 2)(A.暴力，B.优先队列，C．dp乱搞)

A. Carrot Cakes time limit per test：1 second memory limit per test：256 megabytes...

### 活动推荐 