如何在python中使用sklearn中的gridsearchcv进行特征选择
如何在python中使用sklearn中的gridsearchcv进行特征选择
提问于 2019-04-10 17:36:31
我使用recursive feature elimination with cross validation (rfecv)作为randomforest classifier的特性选择器，如下所示。
X = df[[my_features]] #all my features
y = df['gold_standard'] #labels
clf = RandomForestClassifier(random_state = 42, class_weight="balanced")
rfecv = RFECV(estimator=clf, step=1, cv=StratifiedKFold(10), scoring='roc_auc')
rfecv.fit(X,y)
print("Optimal number of features : %d" % rfecv.n_features_)
features=list(X.columns[rfecv.support_])我还按如下方式执行GridSearchCV，以如下方式调优RandomForestClassifier的超参数。
X = df[[my_features]] #all my features
y = df['gold_standard'] #labels
x_train, x_test, y_train, y_test = train_test_split(X, y, random_state=0)
rfc = RandomForestClassifier(random_state=42, class_weight = 'balanced')
param_grid = {
'n_estimators': [200, 500],
'max_features': ['auto', 'sqrt', 'log2'],
'max_depth' : [4,5,6,7,8],
'criterion' :['gini', 'entropy']
}
k_fold = StratifiedKFold(n_splits=10, shuffle=True, random_state=0)
CV_rfc = GridSearchCV(estimator=rfc, param_grid=param_grid, cv= k_fold, scoring = 'roc_auc')
CV_rfc.fit(x_train, y_train)
print(CV_rfc.best_params_)
print(CV_rfc.best_score_)
print(CV_rfc.best_estimator_)
pred = CV_rfc.predict_proba(x_test)[:,1]
print(roc_auc_score(y_test, pred))但是，我不清楚如何将feature selection (rfecv)与GridSearchCV合并。
编辑：
当我运行@Gambit建议的答案时，我得到了以下错误：
ValueError: Invalid parameter criterion for estimator RFECV(cv=StratifiedKFold(n_splits=10, random_state=None, shuffle=False),
estimator=RandomForestClassifier(bootstrap=True, class_weight='balanced',
criterion='gini', max_depth=None, max_features='auto',
max_leaf_nodes=None, min_impurity_decrease=0.0,
min_impurity_split=None, min_samples_leaf=1,
min_samples_split=2, min_weight_fraction_leaf=0.0,
n_estimators='warn', n_jobs=None, oob_score=False,
random_state=42, verbose=0, warm_start=False),
min_features_to_select=1, n_jobs=None, scoring='roc_auc', step=1,
verbose=0). Check the list of available parameters with `estimator.get_params().keys()`.我可以通过在param_grid参数列表中使用estimator__来解决上述问题。
我现在的问题是，如何使用x_test 中的选定功能和参数来验证模型是否能很好地处理未见的数据。如何获取并使用best features optimal hyperparameters**?**训练best features
如果需要，我很乐意提供更多的细节。
回答 3
Stack Overflow用户
回答已采纳
发布于 2019-04-11 17:57:05
基本上，在使用递归特征消除(使用交叉验证)进行特征选择之后，您希望微调分类器的超参数(使用交叉验证)。
流水线对象就是用来组装数据转换和应用估计器的。
也许您可以使用不同的模型(GradientBoostingClassifier等)用于你的最终分类。使用以下方法是可能的：
from sklearn.datasets import load_breast_cancer
from sklearn.feature_selection import RFECV
from sklearn.model_selection import GridSearchCV
from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestClassifier
X, y = load_breast_cancer(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y,
test_size=0.33,
random_state=42)
from sklearn.pipeline import Pipeline
#this is the classifier used for feature selection
clf_featr_sele = RandomForestClassifier(n_estimators=30,
random_state=42,
class_weight="balanced")
rfecv = RFECV(estimator=clf_featr_sele,
step=1,
cv=5,
scoring = 'roc_auc')
#you can have different classifier for your final classifier
clf = RandomForestClassifier(n_estimators=10,
random_state=42,
class_weight="balanced")
CV_rfc = GridSearchCV(clf,
param_grid={'max_depth':[2,3]},
cv= 5, scoring = 'roc_auc')
pipeline = Pipeline([('feature_sele',rfecv),
('clf_cv',CV_rfc)])
pipeline.fit(X_train, y_train)
pipeline.predict(X_test)现在，您可以将此管道(包括功能选择)应用于测试数据。
Stack Overflow用户
发布于 2019-04-11 15:41:11
您可以通过在要传递给估计器的参数的名称前加上'estimator__'来执行所需的操作。
X = df[[my_features]]
y = df[gold_standard]
clf = RandomForestClassifier(random_state=0, class_weight="balanced")
rfecv = RFECV(estimator=clf, step=1, cv=StratifiedKFold(3), scoring='roc_auc')
param_grid = {
'estimator__n_estimators': [200, 500],
'estimator__max_features': ['auto', 'sqrt', 'log2'],
'estimator__max_depth' : [4,5,6,7,8],
'estimator__criterion' :['gini', 'entropy']
}
k_fold = StratifiedKFold(n_splits=3, shuffle=True, random_state=0)
CV_rfc = GridSearchCV(estimator=rfecv, param_grid=param_grid, cv= k_fold, scoring = 'roc_auc')
X_train, X_test, y_train, y_test = train_test_split(X, y)
CV_rfc.fit(X_train, y_train)我做的假数据的输出：
{'estimator__n_estimators': 200, 'estimator__max_depth': 6, 'estimator__criterion': 'entropy', 'estimator__max_features': 'auto'}
0.5653035605690997
RFECV(cv=StratifiedKFold(n_splits=3, random_state=None, shuffle=False),
estimator=RandomForestClassifier(bootstrap=True, class_weight='balanced',
criterion='entropy', max_depth=6, max_features='auto',
max_leaf_nodes=None, min_impurity_decrease=0.0,
min_impurity_split=None, min_samples_leaf=1,
min_samples_split=2, min_weight_fraction_leaf=0.0,
n_estimators=200, n_jobs=None, oob_score=False, random_state=0,
verbose=0, warm_start=False),
min_features_to_select=1, n_jobs=None, scoring='roc_auc', step=1,
verbose=0)Stack Overflow用户
发布于 2019-04-10 18:05:04
您只需将递归特征消除估计器直接传递给GridSearchCV对象即可。像这样的东西应该是可行的
X = df[my_features] #all my features
y = df['gold_standard'] #labels
clf = RandomForestClassifier(random_state = 42, class_weight="balanced")
rfecv = RFECV(estimator=clf, step=1, cv=StratifiedKFold(10), scoring='auc_roc')
param_grid = {
'n_estimators': [200, 500],
'max_features': ['auto', 'sqrt', 'log2'],
'max_depth' : [4,5,6,7,8],
'criterion' :['gini', 'entropy']
}
k_fold = StratifiedKFold(n_splits=10, shuffle=True, random_state=0)
#------------- Just pass your RFECV object as estimator here directly --------#
CV_rfc = GridSearchCV(estimator=rfecv, param_grid=param_grid, cv= k_fold, scoring = 'roc_auc')
CV_rfc.fit(x_train, y_train)
print(CV_rfc.best_params_)
print(CV_rfc.best_score_)
print(CV_rfc.best_estimator_)页面原文内容由Stack Overflow提供。腾讯云小微IT领域专用引擎提供翻译支持
原文链接：
https://stackoverflow.com/questions/55609339
复制相关文章
相似问题