以下文章来源于Datawhale ,作者望尼玛
Hello,大家好。我是“摸鱼打比赛”队的wangli,首先介绍下自己吧,一枚半路出家的野生算法工程师。之所以起名字叫摸鱼打比赛,是因为当时5/6月份自己还处于业务交接没那么忙的一个状态中,然后想起自己也已经毕业两年,但对赛圈一直还是比较关注的,平日看到一些题目也会手痒,但奈何打工人下班之后惰性使然只想躺平,毕业之后始终没有好好打一场比赛,偶尔也会在深夜里问起自己:“廉颇老矣,尚能饭否”,就想着,这回我就利用下这段尚且不忙的日子好好打一场比赛吧。于是我就参加了这次的比赛,不仅侥幸获得了车贷这个小比赛的第一,然后还结识了一些好友,比如我尚在读研的队友陈兄,以及忙于秋招中的好友崔兄。真是收获满满~
那么,接下来我就给大家介绍一下这场比赛中,我的一些具体的解题思路和感悟。
赛题链接:https://challenge.xfyun.cn/topic/info?type=car-loan
可以看到,这个赛题做的是车贷违约预测问题,基于,参赛选手们需要建立风险识别模型来预测可能违约的借款人。这道赛题,相比其他赛题,车贷违约预测这道题的难度是没那么大的,原因有二:
我个人是前期在打商品推荐赛(同“摸鱼打比赛”ID)的时候顺便打一下这个比赛,在最后几天有认真去挖了一些特征。(说到这个基于用户画像的商品推荐赛,就有点惭愧,前期感觉自己还是可以一战的,一度是在Top 3的,后面8月开始由于工作太忙,复赛开始之后就一直没有提交,说到底还是自己时间管理能力太菜了。就看看国庆期间能不能有时间再做一下吧)
再说回这个比赛:
所以,其实可以很快的写出一个baseline来,对于数据新手来说,是一个比较友好的比赛了。
这种偏数据挖掘的比赛的关键点在于如何基于对数据的理解抽象归纳出有用的特征,因此,我一开始做的时候,并没有想着说去套各种高大上的模型,而是通过对数据的分析去构造一些特征。如果不想往后看代码的话,我在这一章节会简单把我的整个方案讲一下:
首先我一开始就发现有很多ID类的特征,然后我就基于这些ID类特征做了一些target encoding特征,这些简单的特征 + 树模型就已经0.583了,能让我前期一直处在Top 10;
而后,从业务角度构造了一些诸如:主账户和二级账户的年利率特征(因为往往银行的利率表现了其对用户的信用预测);从数据分布角度对一些金额类的特征做了些分箱操作;再从特征本身的有效性和冗余角度出发,剔除了一些毫无信息量的特征,比如贷款日期等。这时,我们可以做到0.587这样的水平;
然后,在一次误打误撞的模型训练时,我误把客户ID放进模型中去训练了,结果我发现似乎还对模型性能有一定提升?那我这时候的想法是:这一定是由于欺诈有些集中性导致的,黑产可能在借贷银行(where)或借贷时间(when)上存在一定的集中性,而这种集中性一方面可以通过branch_id/supplier_id/manufacturer_id等反映出来,另一方面,本身客户的customer_id也是可以体现时间上的集中性,因此,我又基于这个点构造了近邻欺诈特征,这时候我们就能做到0.589了;
前期,我一直是用的LightGBM,然后也没有很仔细的去调参(比如hyperopt/ optuna等工具,我都没有用),就很随意(平平无奇的手动调参小天才)
后期,我开始尝试其他的XGBoost/CatBoost/TabNet等模型,但是发现CatBoost和TabNet效果都不是很好,就没有深入往下去钻了(主要白天还是要上班的,因此精力有限,说是摸鱼打比赛,但更准确的说是 熬夜打比赛)
其实主要思路和方案,就如同上述文字所描述的了。但看起来总是干巴巴的,如果你还对代码有兴趣的话,可以继续往下看。毕竟 Talk is Cheap, :)
# 用来TG编码的特征:
TARGET_ENCODING_FETAS = [
'employment_type',
'branch_id',
'supplier_id',
'manufacturer_id',
'area_id',
'employee_code_id',
'asset_cost_bin'
]
# 具体实现:
def gen_target_encoding_feats(train, test, encode_cols, target_col, n_fold=10):
'''生成target encoding特征'''
# for training set - cv
tg_feats = np.zeros((train.shape[0], len(encode_cols)))
kfold = StratifiedKFold(n_splits=n_fold, random_state=1024, shuffle=True)
for _, (train_index, val_index) in enumerate(kfold.split(train[encode_cols], train[target_col])):
df_train, df_val = train.iloc[train_index], train.iloc[val_index]
for idx, col in enumerate(encode_cols):
target_mean_dict = df_train.groupby(col)[target_col].mean()
df_val[f'{col}_mean_target'] = df_val[col].map(target_mean_dict)
tg_feats[val_index, idx] = df_val[f'{col}_mean_target'].values
for idx, encode_col in enumerate(encode_cols):
train[f'{encode_col}_mean_target'] = tg_feats[:, idx]
# for testing set
for col in encode_cols:
target_mean_dict = train.groupby(col)[target_col].mean()
test[f'{col}_mean_target'] = test[col].map(target_mean_dict)
return train, test
def gen_new_feats(train, test):
'''生成新特征:如年利率/分箱等特征'''
# Step 1: 合并训练集和测试集
data = pd.concat([train, test])
# Step 2: 具体特征工程
# 计算二级账户的年利率
data['sub_Rate'] = (data['sub_account_monthly_payment'] * data['sub_account_tenure'] - data[
'sub_account_sanction_loan']) / data['sub_account_sanction_loan']
# 计算主账户的年利率
data['main_Rate'] = (data['main_account_monthly_payment'] * data['main_account_tenure'] - data[
'main_account_sanction_loan']) / data['main_account_sanction_loan']
# 对部分特征进行分箱操作
# 等宽分箱
loan_to_asset_ratio_labels = [i for i in range(10)]
data['loan_to_asset_ratio_bin'] = pd.cut(data["loan_to_asset_ratio"], 10, labels=loan_to_asset_ratio_labels)
# 等频分箱
data['asset_cost_bin'] = pd.qcut(data['asset_cost'], 10, labels=loan_to_asset_ratio_labels)
# 自定义分箱
amount_cols = [
'total_monthly_payment',
'main_account_sanction_loan',
'main_account_disbursed_loan',
'sub_account_sanction_loan',
'sub_account_disbursed_loan',
'main_account_monthly_payment',
'sub_account_monthly_payment',
'total_sanction_loan'
]
amount_labels = [i for i in range(10)]
for col in amount_cols:
total_monthly_payment_bin = [-1, 5000, 10000, 30000, 50000, 100000, 300000, 500000, 1000000, 3000000, data[col].max()]
data[col + '_bin'] = pd.cut(data[col], total_monthly_payment_bin, labels=amount_labels).astype(int)
# Step 3: 返回包含新特征的训练集 & 测试集
return data[data['loan_default'].notnull()], data[data['loan_default'].isnull()]
def gen_neighbor_feats(train, test):
'''产生近邻欺诈特征'''
if not os.path.exists('../user_data/neighbor_default_probs.pkl'):
# 该特征需要跑的时间较久,因此将其存成了pkl文件
neighbor_default_probs = []
for i in tqdm(range(train.customer_id.max())):
if i >= 10 and i < 199706:
customer_id_neighbors = list(range(i - 10, i)) + list(range(i + 1, i + 10))
elif i < 199706:
customer_id_neighbors = list(range(0, i)) + list(range(i + 1, i + 10))
else:
customer_id_neighbors = list(range(i - 10, i)) + list(range(i + 1, 199706))
customer_id_neighbors = [customer_id_neighbor for customer_id_neighbor in customer_id_neighbors if
customer_id_neighbor in train.customer_id.values.tolist()]
neighbor_default_prob = train.set_index('customer_id').loc[customer_id_neighbors].loan_default.mean()
neighbor_default_probs.append(neighbor_default_prob)
df_neighbor_default_prob = pd.DataFrame({'customer_id': range(0, train.customer_id.max()),
'neighbor_default_prob': neighbor_default_probs})
save_pkl(df_neighbor_default_prob, '../user_data/neighbor_default_probs.pkl')
else:
df_neighbor_default_prob = load_pkl('../user_data/neighbor_default_probs.pkl')
train = pd.merge(left=train, right=df_neighbor_default_prob, on='customer_id', how='left')
test = pd.merge(left=test, right=df_neighbor_default_prob, on='customer_id', how='left')
return train, test
最终我只选取了47维特征:
USED_FEATS = [
'customer_id',
'neighbor_default_prob',
'disbursed_amount',
'asset_cost',
'branch_id',
'supplier_id',
'manufacturer_id',
'area_id',
'employee_code_id',
'credit_score',
'loan_to_asset_ratio',
'year_of_birth',
'age',
'sub_Rate',
'main_Rate',
'loan_to_asset_ratio_bin',
'asset_cost_bin',
'employment_type_mean_target',
'branch_id_mean_target',
'supplier_id_mean_target',
'manufacturer_id_mean_target',
'area_id_mean_target',
'employee_code_id_mean_target',
'asset_cost_bin_mean_target',
'credit_history',
'average_age',
'total_disbursed_loan',
'main_account_disbursed_loan',
'total_sanction_loan',
'main_account_sanction_loan',
'active_to_inactive_act_ratio',
'total_outstanding_loan&##39;,
'main_account_outstanding_loan',
'Credit_level',
'outstanding_disburse_ratio',
'total_account_loan_no',
'main_account_tenure',
'main_account_loan_no',
'main_account_monthly_payment',
'total_monthly_payment',
'main_account_active_loan_no',
'main_account_inactive_loan_no',
'sub_account_inactive_loan_no',
'enquirie_no',
'main_account_overdue_no',
'total_overdue_no',
'last_six_month_defaulted_no'
]
def train_lgb_kfold(X_train, y_train, X_test, n_fold=5):
'''train lightgbm with k-fold split'''
gbms = []
kfold = StratifiedKFold(n_splits=n_fold, random_state=1024, shuffle=True)
oof_preds = np.zeros((X_train.shape[0],))
test_preds = np.zeros((X_test.shape[0],))
for fold, (train_index, val_index) in enumerate(kfold.split(X_train, y_train)):
logging.info(f'############ fold {fold} ###########')
X_tr, X_val, y_tr, y_val = X_train.iloc[train_index], X_train.iloc[val_index], y_train[train_index], y_train[val_index]
dtrain = lgb.Dataset(X_tr, y_tr)
dvalid = lgb.Dataset(X_val, y_val, reference=dtrain)
params = {
'objective': 'binary',
'metric': 'auc',
'num_leaves': 64,
'learning_rate': 0.02,
'min_data_in_leaf': 150,
'feature_fraction': 0.8,
'bagging_fraction': 0.7,
'n_jobs': -1,
'seed': 1024
}
gbm = lgb.train(params,
dtrain,
num_boost_round=1000,
valid_sets=[dtrain, dvalid],
verbose_eval=50,
early_stopping_rounds=20)
oof_preds[val_index] = gbm.predict(X_val, num_iteration=gbm.best_iteration)
test_preds += gbm.predict(X_test, num_iteration=gbm.best_iteration) / kfold.n_splits
gbms.append(gbm)
return gbms, oof_preds, test_preds
def train_xgb_kfold(X_train, y_train, X_test, n_fold=5):
'''train xgboost with k-fold split'''
gbms = []
kfold = StratifiedKFold(n_splits=10, random_state=1024, shuffle=True)
oof_preds = np.zeros((X_train.shape[0],))
test_preds = np.zeros((X_test.shape[0],))
for fold, (train_index, val_index) in enumerate(kfold.split(X_train, y_train)):
logging.info(f'############ fold {fold} ###########')
X_tr, X_val, y_tr, y_val = X_train.iloc[train_index], X_train.iloc[val_index], y_train[train_index], y_train[val_index]
dtrain = xgb.DMatrix(X_tr, y_tr)
dvalid = xgb.DMatrix(X_val, y_val)
dtest = xgb.DMatrix(X_test)
params={
'booster':'gbtree',
'objective': 'binary:logistic',
'eval_metric': ['logloss', 'auc'],
'max_depth': 8,
'subsample':0.9,
'min_child_weight': 10,
'colsample_bytree':0.85,
'lambda': 10,
'eta': 0.02,
'seed': 1024
}
watchlist = [(dtrain, 'train'), (dvalid, 'test')]
gbm = xgb.train(params,
dtrain,
num_boost_round=1000,
evals=watchlist,
verbose_eval=50,
early_stopping_rounds=20)
oof_preds[val_index] = gbm.predict(dvalid, iteration_range=(0, gbm.best_iteration))
test_preds += gbm.predict(dtest, iteration_range=(0, gbm.best_iteration)) / kfold.n_splits
gbms.append(gbm)
return gbms, oof_preds, test_preds
def gen_submit_file(df_test, test_preds, thres, save_path):
df_test['test_preds_binary'] = np.where(test_preds > thres, 1, 0)
df_test_submit = df_test[['customer_id', 'test_preds_binary']]
df_test_submit.columns = ['customer_id', 'loan_default']
print(f'saving result to: {save_path}')
df_test_submit.to_csv(save_path, index=False)
print('done!')
return df_test_submit
def gen_thres_new(df_train, oof_preds):
df_train['oof_preds'] = oof_preds
quantile_point = df_train['loan_default'].mean()
thres = df_train['oof_preds'].quantile(1 - quantile_point)
_thresh = []
for thres_item in np.arange(thres - 0.2, thres + 0.2, 0.01):
_thresh.append(
[thres_item, f1_score(df_train['loan_default'], np.where(oof_preds > thres_item, 1, 0), average='macro')])
_thresh = np.array(_thresh)
best_id = _thresh[:, 1].argmax()
best_thresh = _thresh[best_id][0]
print("阈值: {}\n训练集的f1: {}".format(best_thresh, _thresh[best_id][1]))
return best_thresh
# 结果
df_oof_res = pd.DataFrame({'customer_id': train['customer_id'],
'oof_preds_xgb': oof_preds_xgb,
'oof_preds_lgb': oof_preds_lgb,
'loan_default': train['loan_default']
})
# 模型融合
df_oof_res['xgb_rank'] = df_oof_res['oof_preds_xgb'].rank(pct=True)
df_oof_res['lgb_rank'] = df_oof_res['oof_preds_lgb'].rank(pct=True)
df_oof_res['preds'] = 0.31 * df_oof_res['xgb_rank'] + 0.69 * df_oof_res['lgb_rank']
# 得到最优阈值
thres = gen_thres_new(df_oof_res, df_oof_res['preds'])
df_test_res = pd.DataFrame({'customer_id': test['customer_id'],
'test_preds_xgb': test_preds_xgb,
'test_preds_lgb': test_preds_lgb})
df_test_res['xgb_rank'] = df_test_res['test_preds_xgb'].rank(pct=True)
df_test_res['lgb_rank'] = df_test_res['test_preds_lgb'].rank(pct=True)
df_test_res['preds'] = 0.31 * df_test_res['xgb_rank'] + 0.69 * df_test_res['lgb_rank']
# 结果产出
df_submit = gen_submit_file(df_test_res, df_test_res['preds'], thres,
save_path='../prediction_result/result.csv')
Github地址:
https://github.com/WangliLin/xunfei2021_car_loan_top1
结果复现直接运行sh test.sh 即可。