# R in action读书笔记（11）-第八章：回归-- 选择“最佳”的回归模型

8.6 选择“最佳”的回归模型

8.6.1 模型比较

> states<-as.data.frame(state.x77[,c("Murder","Population","Illiteracy","Income","Frost")])

> fit1<-lm(Murder~Population+Illiteracy+Income+Frost,data=states)

>fit2<-lm(Murder~Population+Illiteracy,data=states)

> anova(fit2,fit1)

Analysis of Variance Table

Model 1: Murder ~ Population + Illiteracy

Model 2: Murder ~ Population + Illiteracy + Income +Frost

Res.Df RSS Df Sum of Sq F Pr(>F)

1 47289.25

2 45289.17 2 0.078505 0.0061 0.9939

AIC（AkaikeInformation Criterion，赤池信息准则）也可以用来比较模型，它考虑了模型的

> AIC(fit1,fit2)

df AIC

fit1 6 241.6429

fit2 4 237.6565

8.6.2变量选择

1. 逐步回归stepwise method

），结合了向前逐步回归和向后逐步回归的方法，变量每次进入一个，但是每一步

> library(MASS)

>fit1<-lm(Murder~Population+Illiteracy+Income+Frost,data=states)

>stepAIC(fit1,direction="backward")

Start: AIC=97.75

Murder ~ Population +Illiteracy + Income + Frost

Df Sum of Sq RSS AIC

- Frost 1 0.021 289.19 95.753

- Income 1 0.057 289.22 95.759

<none> 289.17 97.749

- Population 1 39.238 328.41 102.111

- Illiteracy 1 144.264 433.43 115.986

Step: AIC=95.75

Murder ~ Population +Illiteracy + Income

Df Sum of Sq RSS AIC

- Income 1 0.057 289.25 93.763

<none> 289.19 95.753

- Population 1 43.658332.85 100.783

- Illiteracy 1 236.196 525.38 123.605

Step: AIC=93.76

Murder ~ Population +Illiteracy

Df Sum of Sq RSS AIC

<none> 289.25 93.763

- Population 1 48.517 337.76 99.516

- Illiteracy 1 299.646588.89 127.311

Call:

lm(formula = Murder ~Population + Illiteracy, data = states)

Coefficients:

(Intercept) Population Illiteracy

1.6515497 0.0002242 4.0807366

2. 全子集回归

Mallows Cp统计量等准则来选择“最佳”模型

> library("leaps", lib.loc="d:/ProgramFiles/R/R-3.1.3/library")

>leaps<-regsubsets(Murder~Population+Illiteracy+Income+Frost,data=states,nbest=4)

> library(car)

> subsets(leaps,statistic="cp",main="cpplot for all subsets regression")

> abline(1,1,lty=2,col="red")

8.7 深层次分析

8.7.1 交叉验证

fit<-lm(mpg~hp+wt+hp:wt,data=mtcars)

shrinkage<-function(fit,k=10){

require(bootstrap)

theta.fit<-function(x,y){lsfit(x,y)}

theta.predict<-function(fit,x){cbind(1,x)%*%fit\$coef}

x<-fit\$model[,2:ncol(fit\$model)]

y<-fit\$model[,1]

results<-crossval(x,y,theta.fit,theta.predict,ngroup=k)

r2<-cor(y,fit\$fitted.values)^2

r2cv<-cor(y,results\$cv.fit)^2

cat("original r-square=",r2,"\n")

cat(k,"fold cross-validated r-square =",r2cv,"\n")

cat("change=",r2-r2cv),"\n")

}

0 条评论