【教学赛】金融数据分析赛题1:银行客户认购产品预测(0.9676)
阿里云国内75折 回扣 微信号:monov8 |
阿里云国际,腾讯云国际,低至75折。AWS 93折 免费开户实名账号 代冲值 优惠多多 微信号:monov8 飞机:@monov6 |
本文是对天池教学赛银行客户认购产品预测的记录教学赛网址如下
【教学赛】金融数据分析赛题1银行客户认购产品预测_学习赛_天池大赛-阿里云天池
1. 读取数据
import pandas as pd
# 加载数据
train = pd.read_csv('train.csv')
test = pd.read_csv('test.csv')
2. 数据处理
2.1 合并数据
# 训练集和测试集合并, 以便于处理特征的数据
df = pd.concat([train, test], axis=0) #将训练数据和测试数据在行的方向拼接
df
得到的结果
id age job marital education default housing loan contact month ... campaign pdays previous poutcome emp_var_rate cons_price_index cons_conf_index lending_rate3m nr_employed subscribe
0 1 51 admin. divorced professional.course no yes yes cellular aug ... 1 112 2 failure 1.4 90.81 -35.53 0.69 5219.74 no
1 2 50 services married high.school unknown yes no cellular may ... 1 412 2 nonexistent -1.8 96.33 -40.58 4.05 4974.79 yes
2 3 48 blue-collar divorced basic.9y no no no cellular apr ... 0 1027 1 failure -1.8 96.33 -44.74 1.50 5022.61 no
3 4 26 entrepreneur single high.school yes yes yes cellular aug ... 26 998 0 nonexistent 1.4 97.08 -35.55 5.11 5222.87 yes
4 5 45 admin. single university.degree no no no cellular nov ... 1 240 4 success -3.4 89.82 -33.83 1.17 4884.70 no
... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ...
7495 29996 49 admin. unknown university.degree unknown yes yes telephone apr ... 50 302 1 failure -1.8 95.77 -40.50 3.86 5058.64 NaN
7496 29997 34 blue-collar married basic.4y no no no cellular jul ... 8 440 3 failure 1.4 90.59 -47.29 1.77 5156.70 NaN
7497 29998 50 retired single basic.4y no yes no cellular jun ... 3 997 0 nonexistent -2.9 97.42 -39.69 1.29 5116.80 NaN
7498 29999 31 technician married professional.course no no no cellular aug ... 3 1028 0 nonexistent 1.4 96.90 -37.68 5.18 5144.45 NaN
7499 30000 46 admin. divorced university.degree no yes no cellular aug ... 2 387 3 success 1.4 97.49 -31.54 3.79 5082.25 NaN
30000 rows × 22 columns
可见数据既有数字也有文字需要将文字转换为数字
2.2 将非数字的特征转换为数字
# 首先选出所有的特征为object(非数字)的特征
cat_columns = df.select_dtypes(include='object').columns #选择非数字的列对其进行处理
df[cat_columns]
# 对非数字特征进行编码
from sklearn.preprocessing import LabelEncoder
job_le = LabelEncoder()
df['job'] = job_le.fit_transform(df['job'])
df['marital'] = df['marital'].map({'unknown':0, 'single':1, 'married':2, 'divorced':3})
df['education'] = df['education'].map({'unknown':0, 'basic.4y':1, 'basic.6y':2, 'basic.9y':3, 'high.school':4, 'university.degree':5, 'professional.course':6, 'illiterate':7})
df['housing'] = df['housing'].map({'unknown': 0, 'no': 1, 'yes': 2})
df['loan'] = df['loan'].map({'unknown': 0, 'no': 1, 'yes': 2})
df['contact'] = df['contact'].map({'cellular': 0, 'telephone': 1})
df['day_of_week'] = df['day_of_week'].map({'mon': 0, 'tue': 1, 'wed': 2, 'thu': 3, 'fri': 4})
df['poutcome'] = df['poutcome'].map({'nonexistent': 0, 'failure': 1, 'success': 2})
df['default'] = df['default'].map({'unknown': 0, 'no': 1, 'yes': 2})
df['month'] = df['month'].map({'mar': 3, 'apr': 4, 'may': 5, 'jun': 6, 'jul': 7, 'aug': 8, \
'sep': 9, 'oct': 10, 'nov': 11, 'dec': 12})
df['subscribe'] = df['subscribe'].map({'no': 0, 'yes': 1})
2.3 切分数据
# 将数据集重新划分为训练集和测试集 通过subscribe是不是空来判断
train = df[df['subscribe'].notnull()]
test = df[df['subscribe'].isnull()]
# 查看训练集中标签为0和1的比例可以看出0和1不均衡0是1的6.6倍
train['subscribe'].value_counts()
得到
0.0 19548
1.0 2952
Name: subscribe, dtype: int64
2.4 分析数据
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
import warnings
warnings.filterwarnings('ignore')
%matplotlib inline
num_features = [x for x in train.columns if x not in cat_columns and x!='id']
fig = plt.figure(figsize=(80,60))
for i in range(len(num_features)):
plt.subplot(7,2,i+1)
sns.boxplot(train[num_features[i]])
plt.ylabel(num_features[i], fontsize=36)
plt.show()
存在离群点对离群点进行处理
2.5 处理离群点
for colum in num_features:
temp = train[colum]
q1 = temp.quantile(0.25)
q2 = temp.quantile(0.75)
delta = (q2-q1) * 10
train[colum] = np.clip(temp, q1-delta, q2+delta)
## 将超过10倍的值进行处理
2.6 其他处理
进行数据均衡和特征选择但是做完处理后都导致了分类效果变差此处省略。但是把原码贴出来供参考。
'''# 采用SMOTE进行过采样虽然训练的效果好了但是对于最终的分类效果反而降低了此处先不采用过采样
from imblearn.over_sampling import SMOTE
from imblearn.over_sampling import ADASYN
#smo = SMOTE(random_state=0, k_neighbors=10)
adasyn = ADASYN()
X_smo, y_smo = adasyn.fit_resample(train.iloc[:,:-1], train.iloc[:,-1])
train_smo = pd.concat([X_smo, y_smo], axis=1)
train_smo['subscribe'].value_counts()'''
'''# 特征选择方法采用SelectFromModelModel选择树模型
from sklearn.ensemble import ExtraTreesClassifier
from sklearn.feature_selection import SelectFromModel
# 提取出训练数据和标签
train_X = train.iloc[:,:-1]
train_y = train.iloc[:,-1]
# clf_ect是模型名FeaSel为特征选择模型
clf_etc = ExtraTreesClassifier(n_estimators=50)
clf_etc = clf_etc.fit(train_X, train_y)
FeaSel = SelectFromModel(clf_etc, prefit=True)
train_sel = FeaSel.transform(train_X)
test_sel = FeaSel.transform(test.iloc[:,:-1])
# 提取特征名并把特征名写回原始数据
train_new = pd.DataFrame(train_sel)
feature_idx = FeaSel.get_support() #提取选择的列名
train_new.columns = train_X.columns[feature_idx] #将列名写回选择后的数据
train_new = pd.concat([train_new, train_y],axis=1)
test_new = pd.DataFrame(test_sel)
test_new.columns = train_X.columns[feature_idx]'''
此部门内容可能存在变量命名方面的问题。
2.7 数据保存
train_new = train
test_new = test
# 将处理完的数据写回到train_new和test_new进行保存
train_new.to_csv('train_new.csv', index=False)
test_new.to_csv('test_new.csv', index=False)
3. 模型训练
3.1 导入包和数据
from sklearn.model_selection import GridSearchCV
from sklearn.linear_model import LogisticRegression
from sklearn.tree import DecisionTreeClassifier
from sklearn.ensemble import RandomForestClassifier
from sklearn.ensemble import GradientBoostingClassifier
from sklearn.ensemble import AdaBoostClassifier
from xgboost import XGBRFClassifier
from lightgbm import LGBMClassifier
from sklearn.model_selection import cross_val_score
import time
clf_lr = LogisticRegression(random_state=0, solver='lbfgs', multi_class='multinomial')
clf_dt = DecisionTreeClassifier()
clf_rf = RandomForestClassifier()
clf_gb = GradientBoostingClassifier()
clf_adab = AdaBoostClassifier()
clf_xgbrf = XGBRFClassifier()
clf_lgb = LGBMClassifier()
from sklearn.model_selection import train_test_split
train_new = pd.read_csv('train_new.csv')
test_new = pd.read_csv('test_new.csv')
feature_columns = [col for col in train_new.columns if col not in ['subscribe']]
train_data = train_new[feature_columns]
target_data = train_new['subscribe']
3.2 模型调参
from lightgbm import LGBMClassifier
from sklearn.metrics import classification_report
from sklearn.model_selection import GridSearchCV
from sklearn.metrics import accuracy_score
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(train_data, target_data, test_size=0.2,shuffle=True, random_state=2023)
#X_test, X_valid, y_test, y_valid = train_test_split(X_test, y_test, test_size=0.5,shuffle=True,random_state=2023)
n_estimators = [300]
learning_rate = [0.02]#中0.2最优
subsample = [0.6]
colsample_bytree = [0.7] ##在[0.5, 0.6, 0.7]中0.6最优
max_depth = [9, 11, 13] ##在[7, 9, 11, 13]中11最优
is_unbalance = [False]
early_stopping_rounds = [300]
num_boost_round = [5000]
metric = ['binary_logloss']
feature_fraction = [0.6, 0.75, 0.9]
bagging_fraction = [0.6, 0.75, 0.9]
bagging_freq = [2, 4, 5, 8]
lambda_l1 = [0, 0.1, 0.4, 0.5]
lambda_l2 = [0, 10, 15, 35]
cat_smooth = [1, 10, 15, 20]
param = {'n_estimators':n_estimators,
'learning_rate':learning_rate,
'subsample':subsample,
'colsample_bytree':colsample_bytree,
'max_depth':max_depth,
'is_unbalance':is_unbalance,
'early_stopping_rounds':early_stopping_rounds,
'num_boost_round':num_boost_round,
'metric':metric,
'feature_fraction':feature_fraction,
'bagging_fraction':bagging_fraction,
'lambda_l1':lambda_l1,
'lambda_l2':lambda_l2,
'cat_smooth':cat_smooth}
model = LGBMClassifier()
clf = GridSearchCV(model, param, cv=3, scoring='accuracy', verbose=1, n_jobs=-1)
clf.fit(X_train, y_train, eval_set=[(X_train, y_train),(X_test, y_test)])
print(clf.best_params_, clf.best_score_)
里面只有1个值的是已经通过GridSearchCV找到的最优优值了程序显示的是最后的6个参数的寻优都放到一起训练时间太长了所以选择分开寻找。
得到的结果
Early stopping, best iteration is:
[287] training's binary_logloss: 0.22302 valid_1's binary_logloss: 0.253303
{'bagging_fraction': 0.6, 'cat_smooth': 1, 'colsample_bytree': 0.7, 'early_stopping_rounds': 300, 'feature_fraction': 0.75, 'is_unbalance': False, 'lambda_l1': 0.4, 'lambda_l2': 10, 'learning_rate': 0.02, 'max_depth': 11, 'metric': 'binary_logloss', 'n_estimators': 300, 'num_boost_round': 5000, 'subsample': 0.6} 0.8853333333333334
3.3 预测结果
y_true, y_pred = y_test, clf.predict(X_test)
accuracy = accuracy_score(y_true,y_pred)
print(classification_report(y_true, y_pred))
print('Accuracy',accuracy)
结果
precision recall f1-score support
0.0 0.91 0.97 0.94 3933
1.0 0.60 0.32 0.42 567
accuracy 0.89 4500
macro avg 0.75 0.64 0.68 4500
weighted avg 0.87 0.89 0.87 4500
Accuracy 0.8875555555555555
查看混淆矩阵
from sklearn import metrics
confusion_matrix_result = metrics.confusion_matrix(y_true, y_pred)
plt.figure(figsize=(8,6))
sns.heatmap(confusion_matrix_result, annot=True, cmap='Blues')
plt.xlabel('predict')
plt.ylabel('true')
plt.show()
4. 输出结果
test_x = test[feature_columns]
pred_test = clf.predict(test_x)
result = pd.read_csv('./submission.csv')
subscribe_map ={1: 'yes', 0: 'no'}
result['subscribe'] = [subscribe_map[x] for x in pred_test]
result.to_csv('./baseline_lgb1.csv', index=False)
result['subscribe'].value_counts()
结果
no 6987
yes 513
Name: subscribe, dtype: int64
5. 提交结果
6. 总结
本人的方法只获得了0.9676的结果希望您能在本人的程序基础上进行改进以得到更佳的效果。如果有了更好的方法欢迎在留言区告诉我相互讨论。
改进的思路
1. 数据处理方面本人在进行数据均衡时训练的效果很好但是最终的效果较差应该是数据过拟合了另外在数据的离群点处理方面也可以做更进一步的考虑
2.方法的改进本人对比了lr, dt, rf, gb, adab, xgbrf, lgb最终lgb的效果最好所以最终选择lgb进行调参可以考虑采用多种方法的组合进行训练
3.在lgb的基础上进行调参这个是最没有科技含量的。不过花时间应该会得到比我的结果更好的效果。