和鲸社区2022咸鱼打挺夏令营-机器学习原理与实践·闯关-作业答案与部分解析

Jul 20, 2022

第一关 - 逻辑回归#

作业#

作业1：逻辑回归的表达式：

A: h(x)=wx+b

B：h(x)=wx

C: h(x)=sigmoid(wx+b)

D: h(x)=sigmoid(wx)
作业2：下面关于逻辑回归的表述是正确的(多选)：

A<逻辑回归的输出结果是概率值>，在0-1之间

B<使用正则化可以提高模型的泛化性>

C<逻辑回归可以直接用于多分类>

D<逻辑回归是无参模型>

E<逻辑回归的损失函数是交叉熵>
作业3：计算 $y=sigmoid(w_1*x_1+w_2*x_2+1)$ 当 w=(0.2, 0.3)时，样本X=(1,1),y=1的时w1,w2的梯度和loss：(保存3位小数，四舍五入)

作业4：在cal_grad梯度函数的基础上加上L2正则化，下面的函数是否正确?(Y/N)

1
def cal_grad(y, t,x,w):
2
    """
3
    x:输入X
4
    y:样本y
5
    t:预测t
6
    w:参数w
7
    """
8
    grad = np.sum(t - y) / t.shape[0]
9
    return grad*x+2*w

答案#

	id	answer
0	a1	C
1	a2	ABE
2	a3	-0.182
3	a4	-0.182
4	a5	0.201
5	a6	Y

解析#

t1代码

1
import numpy as np
2

3
def sigmoid(x):
4
    return 1 / (1 + np.exp(-x))
5

6
def cal_grad(y, t):
7
    grad = np.sum(t - y)
8
    return grad
9

10
def cal_cross_loss(y, t):
11
    loss=np.sum(-y * np.log(t)- (1 - y) * np.log(1 - t))
12
    print(loss)
13
    return loss
14

15
def linearRegression(x):
16
    return sum([x[i]*w[i] for i in range(len(x))])+b
17

18
b=1
19
w=np.array([0.2,0.3])
20
x=np.array([1.0,1.0])
21

22
# linearRegression(x)
23
# sigmoid(linearRegression(x))
24
cal_grad(1,sigmoid(linearRegression(x)))
25

26
# 这里去掉了shape[0] 因为只有一组数据

第二关 - 朴素贝叶斯法#

作业#

1<假设a>,B两个盒子球有无限个，已知从A盒子摸出红球和白球的概率为0.7和0.3，从B盒子中摸出的红球和白球的概率为0.5和0.5。从某一个盒子中摸了3次球，颜色依次为白，白，红。问是从A盒子中摸得的概率是多少？(保留4位小数)
2<贝叶斯推断最重要的假设是什么>？

A：独立性假设

B：参数服从一个分布
3<贝叶斯估计与最大似然估计的区别是>：(Y/N)

贝叶斯假设参数服从一个分布,不是一个确定的值，而最大似然估计认为参数是一个值。

4<假设训练数据>：确定 x=(2,S) 的结果，求出y的预测结果

1
X=[["1","S"],["1","M"],["1","M"],["1","S"],["1","S"],\
2
   ["2","S"],["2","M"],["2","M"],["2","L"],["2","L"],\
3
   ["3","L"],["3","M"],["3","M"],["3","L"],["3","L"]]
4
Y=[0,0,1,1,0,0,0,1,1,1,1,1,1,1,0]

5<第四题中>，当使用拉普拉斯平滑，即λ=1，求y的预测结果

答案#

	id	answer
0	a1	0.3351
1	a2	A
2	a3	Y
3	a4	0
4	a5	0

解析#

t1解法

1
p（1|白白红）= p（白白红，1）/p（白白红）
2
           = p（白白红|1）*p（1）/=（p（白白红|1）*p（1）+p（白白红|2）*p（2））
3
           = 0.3*0.3*0.7*0.5/（0.3*0.3*0.7*0.5+0.5*0.5*0.5*0.5）
4
           = 0.335106383

t4解法：看统计学习方法，原题

题目讲解来源：【合集】十分钟机器学习系列视频《统计学习方法》

t5看公式

P(Y=c_k)=\frac{\sum_{i=1}^K{I(y_i=c_k)+\lambda}}{N+K\lambda}

当 $\lambda$ 等于1的时候，称之为拉普拉斯平滑。

p（y=1）=（9+1）/（15+2）

p（y=0）=（6+1）/（15+2）

第三关 - K近邻算法#

作业#

1：请你回顾下KNN算法的三要素：

2：改写一下函数使其可以应用于KNN的回归预测,回归预测的loss为平方差

1
def predict(self, train_x, y, test, k):
2
    """
3
    返回根据KNN预测的结果
4
    :param train_x: 训练集x
5
    :param y: 训练集y
6
    :param test: 预测集
7
    :return: 返回test预测的结果
8
    """
9
    dis = self.euclidean_dis(test, train_x)
10
    k_neighbor = np.argsort(dis, axis=1)[:, :k]
11
    k_neighbor_value = y[k_neighbor]
12
    n = test.shape[0]  # 预测结果的个数
13
    pred = np.zeros(n)
14
    for i in range(n):
15
        pred[i] = np.argmax(np.bincount(k_neighbor_value[i])) ##改写1
16
    return pred
17

18

19
def KFlod(self, k):
20
    folds = StratifiedKFold(n_splits=5, shuffle=True, random_state=1996)
21
    oof = np.zeros(self.train_x.shape[0])
22
    for fold_, (train_index, test_index) in enumerate(folds.split(self.train_x, self.train_y)):
23
        train_x, test_x, train_y, test_y = self.train_x[train_index], self.train_x[test_index], \
24
                                           self.train_y[
25
                                               train_index], self.train_y[test_index]
26
        pred = self.predict(train_x, train_y, test_x, k)
27
        oof[test_index] = pred
28
    return np.sum(oof == self.train_y)                     #改写2

下面那个改写是正确的：

A：pred[i]=np.sum(k_neighbor_value[i]),np.sum(oof - self.train_y)

B: pred[i]=np.mean(k_neighbor_value[i]),np.sum(oof - self.train_y)

C：pred[i]=np.sum(k_neighbor_value[i]),np.sum((oof - self.train_y)**2)

D: pred[i]=np.mean(k_neighbor_value[i]),np.sum((oof - self.train_y)**2)

3<使用全部iris数据>，选取中最优的K值，并且计算此时分类正确的个数

答案#

	id	answer
0	a1	K值
1	a2	度量距离
2	a3	决策规则
3	a4	D
4	a5	13
5	a6	146

解析#

t3代码

1
import numpy as np
2
from sklearn import datasets
3
from sklearn.model_selection import train_test_split
4
from sklearn.model_selection import StratifiedKFold
5

6
X = datasets.load_iris()['data']
7
Y = datasets.load_iris()['target']
8
X_train, X_test, y_train, y_test = train_test_split(X, Y, test_size=0.4, stratify=Y,random_state=100)
9
X_test = datasets.load_iris()['data']
10
y_test = datasets.load_iris()['target']
11

12
class KNN:
13
    def __init__(self, train_x, train_y, test_x, test_y):
14
        """
15
        KNN初始话
16
        :param train_x:训练集X
17
        :param train_y: 训练集Y
18
        :param test_x: 预测集X
19
        :param test_y: 预测集Y
20
        """
21
        self.train_x = train_x
22
        self.train_y = train_y
23
        self.test_x = test_x
24
        self.test_y = test_y
25
        self.k = None
26

27
    def euclidean_dis(self, x1, x2):
28
        """
29
        返回x1与x2的距离(x1,x2均为二维矩阵).x1.shape=(N1*M),x2.shape=(N2*M2),返回结果为(N1*N2)
30
        :param x1:
31
        :param x2:
32
        :return:
33
        """
34
        n1, m1 = x1.shape
35
        n2, m2 = x2.shape
36
        if m1 != m2:
37
            raise ("两个向量维度不相等")
38
        x1x2 = np.dot(x1, x2.T)  # (n1,n2)
39
        y1 = np.repeat(np.reshape(np.sum(np.multiply(x1, x1), axis=1), (n1, 1)), repeats=n2, axis=1)
40
        y2 = np.repeat(np.reshape(np.sum(np.multiply(x2, x2), axis=1), (n2, 1)), repeats=n1, axis=1).T
41
        dis = y1 + y2 - 2 * x1x2
42
        return dis
43

44
    def predict(self, train_x, y, test, k):
45
        """
46
        返回根据KNN预测的结果
47
        :param train_x: 训练集x
48
        :param y: 训练集y
49
        :param test: 预测集
50
        :return: 返回test预测的结果
51
        """
52
        dis = self.euclidean_dis(test, train_x)
53
        k_neighbor = np.argsort(dis, axis=1)[:, :k]
54
        k_neighbor_value = y[k_neighbor]
55
        n = test.shape[0]  # 预测结果的个数
56
        pred = np.zeros(n)
57
        for i in range(n):
58
            pred[i] = np.argmax(np.bincount(k_neighbor_value[i]))
59
        return pred
60

61
    def KFlod(self, k):
62
        folds = StratifiedKFold(n_splits=5, shuffle=True, random_state=1996)
63
        oof = np.zeros(self.train_x.shape[0])
64
        for fold_, (train_index, test_index) in enumerate(folds.split(self.train_x, self.train_y)):
65
            train_x, test_x, train_y, test_y = self.train_x[train_index], self.train_x[test_index], \
66
                                               self.train_y[
67
                                                   train_index], self.train_y[test_index]
68
            pred = self.predict(train_x, train_y, test_x, k)
69
            oof[test_index] = pred
70
        return np.sum(oof == self.train_y)
71

72
    def selectK(self):
73
        ks = [2,3, 4, 5, 6,7,8,9,10,11,12,13,14,15]
74
        value = 0
75
        for k in ks:
76
            value_tem = self.KFlod(k)
77
            print("当前K的值为：", k, "预测得分为：", value_tem)
78
            if value_tem > value:
79
                self.k = k
80
                value = value_tem
81

82
    def trainAndPredic(self):
83
        self.selectK()
84
        print("选择的k为：", self.k)
85
        preds = self.predict(self.train_x, self.train_y, self.test_x, self.k)
86
        print("预测结果的正确个数为:", np.sum(preds == self.test_y))
87
        print("预测结果的错误个数为:", np.sum(preds != self.test_y))
88

89

90
model = KNN(X_train, y_train, X_test, y_test)
91
model.trainAndPredic()
92

93
"""
94
当前K的值为： 2 预测得分为： 89
95
当前K的值为： 3 预测得分为： 89
96
当前K的值为： 4 预测得分为： 89
97
当前K的值为： 5 预测得分为： 89
98
当前K的值为： 6 预测得分为： 88
99
当前K的值为： 7 预测得分为： 89
100
当前K的值为： 8 预测得分为： 89
101
当前K的值为： 9 预测得分为： 89
102
当前K的值为： 10 预测得分为： 89
103
当前K的值为： 11 预测得分为： 89
104
当前K的值为： 12 预测得分为： 89
105
当前K的值为： 13 预测得分为： 90
106
当前K的值为： 14 预测得分为： 88
107
当前K的值为： 15 预测得分为： 89
108
选择的k为： 13
109
预测结果的正确个数为: 146
110
预测结果的错误个数为: 4
111
"""

第四关 - K-means算法#

题目#

数据准备

1
import numpy as np
2
from sklearn import datasets
3
from sklearn.model_selection import train_test_split
4
X=datasets.load_iris()['data']
5
Y=datasets.load_iris()['target']

1.对样本X进行归一化(均值方差归一化)，输出并且将归一化的结果的第一行(将输出的list转化为string,用逗号(,)连接，四舍五入保留3位小数，注意不能有空格)
2<计算上一步>归一化之后样本中第一个样本与最后一个样本的欧式距离（四舍五入3位小数）。
3：根据数据中已有的类别(Y就是类别)，分别计算类别0，类别1，类别2的中心点(不归一化)：(将输出的list转化为string,用逗号(,)连接，四舍五入保留1位小数，不能有空格)

答案#

	id	answer
0	a1	-0.901,1.019,-1.340,-1.315
1	a2	3.335
2	a3	5.0,3.4,1.5,0.2
3	a4	5.9,2.8,4.3,1.3
4	a5	6.6,3.0,5.6,2.0

解析#

题目1

1
x_mean = np.mean(X, axis=0)
2
length = len(X)
3
x_std =  sum((X-x_mean)**2)/length
4
X1 = (X-x_mean)/np.sqrt(x_std)
5
X1[0]

题目2

1
np.sqrt(sum((X1[0]-X1[-1])**2))

题目3（不要看错题目认为是根据kmeans计算得到聚类中心点，而是根据已有标签计算中心点）

1
pos = []
2
pos.append(np.mean(X[Y==0],axis=0))
3
pos.append(np.mean(X[Y==1],axis=0))
4
pos.append(np.mean(X[Y==2],axis=0))
5
for i in pos:
6
    for j in i:
7
        print('{:.1f}'.format(j),end=',')
8
    print("")

第五关 - 线性判别器LDA#

题目#

1：在2.2(扩展到多分类)中说到，“两种计算方式的结果是成比例的”，请你计算一下使用我们代码中计算方法和采用另外一种方法计算两者之间的比例。 $S_B1$ 为我给出代码中的计算方法， $S_B2$ 为 $S_b=\sum_{i,j|i\neq j}[(u_i-u_j)(u_i-u_j)^T]$ ，直接给出 $S_B1$ 与 $S_B2$ 之间的比例。
2：下面关于线性判别器的理解正确的是？（不定项）

A：线性判别器是一个分类器

B：线性判别器是一个无监督模型

C<线性判别器是一个降维方法>

D<线性判别器可以降低至任意维度>

答案#

	id	answer
0	a1	8.33
1	a2	C

解析#

t1代码

1
import numpy as np
2
from sklearn import datasets
3

4
from sklearn.datasets import make_blobs
5
import matplotlib.pyplot as plt
6
from sklearn.model_selection import train_test_split
7

8
X = datasets.load_iris()['data']
9
Y = datasets.load_iris()['target']
10

11

12
class LDA:
13
    def __init__(self, k_after):
14
        """
15
        x:         样本x
16
        y:         样本y
17
        n_i：      第i类样本的个数
18
        u_i：      第i类样本均值,格式为{i:[]}
19
        n_label：  样本的类别
20
        k_after:   降维后的维度
21
        k_before:  降维前的维度]
22
        labels:    不同的类别，比如[1,2,3]
23
        """
24
        self.n_i = {}
25
        self.u_i = {}
26
        self.k_after = k_after
27
        self.k_before = X.shape[0]
28
        self.labels = None
29
        self.sigmas = {}
30
        self.S = None  # S_w
31
        self.B = None  # S_b
32
        self.w = None
33

34
    def fit(self, X, y):
35
        self.n = len(np.unique(y))
36
        self.n_label = len(set(y))
37
        labels = np.unique(y)
38
        self.labels = labels
39
        N = X.shape[0]  # 样本的个数
40
        means = []
41

42
        for label in labels:
43
            tmp = np.mean(X[y == label], axis=0)  ##第i类的平均u_i
44

45
            means.append(tmp)
46
            self.u_i[label] = tmp  ##记录第i类样本的均值
47
            self.n_i[label] = len(X[y == label])   #记录第i类样本的个数
48

49
        if len(labels) == 2:
50
            tmp = (means[0] - means[1])
51
            tmp = tmp.reshape(-1, 1)  # 转化为(k_before,1)维度的列向量
52
            B = np.dot(tmp, tmp.T)  # (u[0]-u[1])(u[0]-u[1])^T
53
        else:
54
            mean_all = np.mean(X, axis=0)
55
            B = np.zeros((X.shape[1], X.shape[1]))
56
            for label in self.u_i:
57
                n_i = self.n_i[label]
58
                tmp = self.u_i[label] - mean_all
59
                tmp = tmp.reshape(-1, 1)
60
                B += n_i * np.dot(tmp, tmp.T)
61
            print('Sb1=',B)
62

63
            Sb2 = np.zeros((X.shape[1], X.shape[1]))
64
            for i in range(len(self.u_i)):
65
                for j in range(len(self.u_i)):
66
                    if i != j:
67
                        tmp = self.u_i[i] - self.u_i[j]
68
                        tmp = tmp.reshape(-1, 1)
69
                        Sb2 += np.dot(tmp, tmp.T)
70
            print('Sb2=',Sb2)
71

72
            print('Sb1/Sb2=',B/Sb2)
73

74
        S = np.zeros((X.shape[1], X.shape[1]))
75
        for label in self.u_i:
76
            u_i = self.u_i[label]
77
            for row in X[y == label]:
78
                tmp = (row - u_i)
79
                tmp = tmp.reshape(-1, 1)
80
                S += np.dot(tmp, tmp.T)
81
        self.S = S
82

83

84

85

86

87
        S_inv = np.linalg.inv(S)  # 矩阵S_w的逆
88
        S_inv_B = S_inv @ B  # S_w*B
89
        diag, p = np.linalg.eig(S_inv_B)  ## 特征值，特征向量
90

91
        ind = diag.argsort()[::-1]  ##按照特征值大小排序
92
        diag = diag[ind]
93
        w = p[:, ind]  # 按照特征值大小将特征向量排序
94
        self.w = w[:, :self.k_after]
95

96
    def predict(self, x):
97
        x = np.asarray(x)
98
        return np.dot(x, self.w)
99

100

101
model = LDA(2)
102
model.fit(X, Y)
103
X2=model.predict(X)
104

105
"""
106
Sb1= [[ 63.21213333 -19.95266667 165.2484      71.27933333]
107
 [-19.95266667  11.34493333 -57.2396     -22.93266667]
108
 [165.2484     -57.2396     437.1028     186.774     ]
109
 [ 71.27933333 -22.93266667 186.774       80.41333333]]
110

111
Sb2= [[ 7.585456 -2.39432  19.829808  8.55352 ]
112
 [-2.39432   1.361392 -6.868752 -2.75192 ]
113
 [19.829808 -6.868752 52.452336 22.41288 ]
114
 [ 8.55352  -2.75192  22.41288   9.6496  ]]
115

116
Sb1/Sb2= [[8.33333333 8.33333333 8.33333333 8.33333333]
117
 [8.33333333 8.33333333 8.33333333 8.33333333]
118
 [8.33333333 8.33333333 8.33333333 8.33333333]
119
 [8.33333333 8.33333333 8.33333333 8.33333333]]
120
"""

第六关 - 支持向量机#

题目#

1：下面逻辑回归与支持向量机的对比正确的是：（多选，注意大写）

A：逻辑回归速度比SVM更快

B：SVM和逻辑回归一样，任何样本都会对最终情况产生影响

C：都不能直接进行多分类

D：逻辑回归输出概率，SVM直接输出类别

E：两者的目标函数不同，逻辑回归是交叉熵，SVM是Hinge Loss
2：关于选取最大分离面下面说法是否正确？(Y/N)

回答：理论上来说，分离超平面有无数个，但是最大分离面只有一个。此外，使用最大间隔分离面对未知的样本有更好的泛化能力。
3：下面关于核函数的介绍正确的是？（多选，注意大写）

A：提升SVM的非线性拟合能力

B：可以提升SVM模型的准确度

C：能够使得模型训练速度变快

D：降低模型过拟合的风险

答案#

	id	answer
0	a1	ACDE
1	a2	Y
2	a3	AB

第七关 - 决策树#

题目#

1：写出文中介绍的三种分类算法(按照介绍的顺序)

2：计算下面数据的熵与Gini指数（保留3位小数，四舍五入）

1
import pandas as pd
2
data_path=r"/home/mw/input/data2794"
3
data = pd.read_csv(r"/home/mw/input/data2794/西瓜数据集.csv")

答案#

	id	answer
0	a1	信息增益
1	a2	信息增益比
2	a3	Gini指数
3	a4	0.998
4	a5	0.498

解析#

t2代码

1
import pandas as pd
2
from math import log
3
def calShanEnt(dataset,col):
4
    tarset=set(dataset[col])
5
    res=0
6
    for i in tarset:
7
        pi=np.sum(dataset[col] == i)/len(dataset)
8
        res=res-pi* log(pi, 2)
9
    return res
10

11
calShanEnt(data,"target")
12

13
def Gini(dataset,col):
14
    tarset = set(dataset[col])
15
    gini=1
16
    for i in tarset:
17
        gini=gini-(np.sum(dataset[col] == i)/len(dataset))**2
18
    return gini
19

20
Gini(data,"target")

第八关 - 基于xgboost的分类预测#

题目#

1：xgb与GBDT在损失函数上的区别(多选)

A：正则化

B：二阶残差

C：一阶残差
2：计算函数 f=x3+x2 在x=0.5处的二阶梯度
3：下面哪些方式可以提升xgb模型的准确度（多选）

A：更多的数据

B：更好的特征工程

C：减少树的深度

D：增加学习率
4：令x=x_train[<1>]，使用上面的sklearn包装的xgb模型预测x所对应得target值(输出target为1概率，保留4位小数)

答案#

	id	answer
0	a1	AB
1	a2	5
2	a3	ABCD
3	a4	0.0307

解析#

t4代码

1
import numpy as np
2
from sklearn import datasets
3
import pandas as pd
4
from sklearn.model_selection import train_test_split
5
X=datasets.load_breast_cancer()['data']
6
Y=datasets.load_breast_cancer()['target']
7
fea=datasets.load_breast_cancer()['feature_names']
8

9
data=pd.DataFrame(X,columns=fea)
10
data['target']=Y
11
x_train=data[fea]
12
y_train=data['target']
13
data['target'].value_counts()
14
from sklearn.model_selection import KFold,StratifiedKFold
15
import xgboost as xgb
16
import datetime
17
import sklearn
18
folds = KFold(n_splits=5, shuffle=True, random_state=1996)
19
params = {
20
    'learning_rate': 0.05,
21
    'max_depth': 8,
22
    'eval_metric': 'auc',
23
    'objective': 'binary:logistic',
24
}
25
import warnings
26
warnings.filterwarnings('ignore')
27
oof=np.zeros(x_train.shape[0])
28
for fold_, (train_index, test_index) in enumerate(folds.split(x_train, y_train)):
29
    print("第{}折".format(fold_))
30

31
    train_x, test_x, train_y, test_y = x_train.iloc[train_index], x_train.iloc[test_index], y_train.iloc[train_index], y_train.iloc[test_index]
32

33
    model=xgb.XGBClassifier(objective="binary:logistic",booster="gbtree"
34
                ,max_depth=6, learning_rate=0.05, n_estimators=1000,
35
                  n_jobs=1, gamma=1,
36
#                  min_child_weight=1, subsample=0.9,
37
#                  colsample_bytree=0.9,
38
#                  colsample_bynode=0.85, reg_alpha=1, reg_lambda=1,
39
                 scale_pos_weight=5, random_state=2022)
40
    model.fit(train_x,train_y,sample_weight=None,eval_set=[(train_x,train_y),(test_x,test_y)],eval_metric="auc",early_stopping_rounds=50,verbose=20,)
41
    val_train=model.predict_proba(test_x)[:,1]
42
    oof[test_index]=val_train
43
print("最终auc为：",sklearn.metrics.roc_auc_score(y_train,oof))
44

45
x=x_train[:1]
46
model.predict_proba(x)
47

48
"""
49
...
50
最终auc为： 0.9810461920617303
51
array([[0.9692602 , 0.03073976]], dtype=float32)
52
"""

关卡9 基于LightGBM的数据实践#

1：下面哪些是lgb相对于xgb的优化点？

A：使用了二阶残差

B：使用了直方图进行加速计算

C：可以支持类别特征处理

D：使用了单边梯度采样减少计算样本

E：使用了互斥捆绑算法进行特征组合减少了特征数量

F：使用了带深度限制的 Leaf-wise 算法避免了无效的树生长
2：假设叶节点的数量相同，xgb的深度一定会比lgb的深度大？(Y/N)
3：下面关于lgb与随机森林的说法正确的是（多选）？

A：lgb和随机森林都是由多颗决策树组成

B：Lgb和随机森林的树都是可以并行生成的

C：随机森林的决策树可以是回归树也可以是分类树，lgb只能是回归树

D：随机森林主要是通过减低方差提高模型的泛化能力，lgb是降低偏差提高模型的拟合能力

答案#

	id	answer
0	a1	BCDEF
1	a2	N
2	a3	ACD

关卡10 BP神经网络#

题目#

1：BP神经网络过拟合时可以通过哪些方法来改进

A：减少网络深度

B：减少神经元的数量

C：增加dropout

D：增加数据量
2：当进行回归预测时，计算输出层的loss和回传梯度

t=np.array([[0.00324988] [0.01669568] [0.01676606] [0.97652019]])

y=np.array([[0],[0],[0],[1]])

提交结果格式为：0.3,0.2,0.2,0.4,0.5 前四个数字为梯度，第五个为loss，保留5位小数(四舍五入，请注意不要有空格)
以上面的神经网络为例，请问一下总共的参数数量为多少？答案为整数
使用上面的神经网络预测test=[[0.3,0.4],[1,2]]的类别。输出结果为(两个结果直接相连，比如提交01，11，10等)

答案#

	id	answer
0	a1	ABCD
1	a2	0.00325,0.01670,0.01677,-0.02348,0.01519
2	a3	17
3	a4	01

解析#

t2代码

1
def cal_grad(y, t):
2
    grad = t - y
3
    return grad
4

5
t=np.array([[0.00324988],[0.01669568],[0.01676606],[0.97652019]])
6
y=np.array([[0],[0],[0],[1]])
7
cal_grad(y,t)
8

9
"""
10
array([[ 0.00324988],
11
       [ 0.01669568],
12
       [ 0.01676606],
13
       [-0.02347981]])
14
"""
15

16
def cal_cross_loss(y, t):
17
    loss=np.sum(-y * np.log(t)- (1 - y) * np.log(1 - t))/t.shape[0]
18
    return loss
19

20

21
cal_cross_loss(y,t)
22

23
"""
24
0.015189962638228426
25
"""

包括w和b

一层2->4 2*4+4=12

一层4->1 4*1+1=5

12+5=17个

t4代码

1
import numpy as np
2
X = np.array([[0.3,0.4],[1,2]])
3
y=np.array([[0],[1]])
4
net=Net()
5
net.__init__(2,4,1,0.1)
6
net.trian(X,y)
7

8
"""
9
[[0.3 0.4]
10
 [1.  2. ]]
11
[[0]
12
 [1]]
13
[[0.02215638]
14
 [0.98925725]]
15
"""

模型

1
class FC:
2
    def init(self, in_num, out_num, lr=0.01):
3
        self._in_num = in_num
4
        self._out_num = out_num
5
        self.w = np.random.rand(in_num, out_num)  # 生成out行，in列矩阵
6

7
        self.b = np.zeros(out_num)
8
        self.lr = lr
9

10
    def _sigmoid(self, in_data):
11
        return 1 / (1 + np.exp(-in_data))
12

13
    def forward(self, in_data):
14
        self.z = np.dot(in_data, self.w, ) + self.b
15
        self.top_val = self._sigmoid(self.z)
16
        self.bottom_val = in_data
17
        return self.top_val
18

19
    def backward(self, loss):
20
        residual_z = loss * self.top_val * (1 - self.top_val)
21

22
        grad_w = np.dot(self.bottom_val.T, residual_z)
23
        grad_b = np.sum(residual_z)
24

25
        self.w -= self.lr * grad_w
26
        self.b -= self.lr * grad_b
27
        residual_x = np.dot(residual_z,self.w.T)
28
        return residual_x
29

30

31
class Net:
32
    def __init__(self, input_num=2, hidden_num=4, out_num=1, lr=0.05):
33
        self.fc1 = FC()
34
        self.fc1.init(input_num, hidden_num, lr) 2-4
35
        self.fc2 = FC()
36
        self.fc2.init(hidden_num, out_num, lr)  4-1
37
        self.loss = Loss()
38

39
    def trian(self, x, y):
40
        for i in range(1000):
41
            layer1out = self.fc1.forward(x)#2-4 1 12
42
            layer2out = self.fc2.forward(layer1out)#4-1
43
            loss = self.loss.forward(y, layer2out)
44
            delta2 = self.loss.backward()
45
            delta1 = self.fc2.backward(delta2)  17
46
            saliency = self.fc1.backward(delta1)
47
        layer1out = self.fc1.forward(X)
48
        layer2out = self.fc2.forward(layer1out)
49
        print(X)
50
        print(y)
51
        print(layer2out)
52

53
net=Net()
54
net.__init__(2,4,1,0.1)
55
net.trian(X,y)

大作业：疾病预测#

使用欠采样处理正负样本不能不均衡（0.816）#

1
## 直接导入之前已经处理好的数据
2
import pandas as pd
3
import re
4
mydata=pd.read_csv('/home/mw/input/MLearn9130/训练营.csv')
5
mytest=pd.read_csv('/home/mw/input/MLearn9130/测试集.csv')
6
## 分离训练集和验证集
7
from sklearn.model_selection import train_test_split
8

9
## 原始列名列表
10
col_names=list(mydata.columns)
11
col=[]
12
for i in range(len(col_names)):
13
    if re.findall(r"\u2028(.+)",col_names[i])!=[]:
14
        col.append(re.findall(r"\u2028(.+)",col_names[i])[0])
15
    elif re.findall(r"\n(.+)",col_names[i])!=[]:
16
        col.append(re.findall(r"\n(.+)",col_names[i])[0])
17
    else:
18
        col.append(col_names[i])
19

20
col_names1=list(mytest.columns)
21
col1=[]
22
for i in range(len(col_names1)):
23
    if re.findall(r"\u2028(.+)",col_names1[i])!=[]:
24
        col1.append(re.findall(r"\u2028(.+)",col_names1[i])[0])
25
    elif re.findall(r"\n(.+)",col_names1[i])!=[]:
26
        col1.append(re.findall(r"\n(.+)",col_names1[i])[0])
27
    else:
28
        col1.append(col_names1[i])
29

30
## 修改dataframe列名
31
mydata.columns=col
32
mytest.columns=col1
33

34

35
## 对性别进行编码
36
def gender(x):
37
    if x=='M':
38
        return 0
39
    else:
40
        return 1
41

42
## 对区域进行编码
43
def district(x):
44
    if x=='east':
45
        return 1
46
    elif x=='south':
47
        return 2
48
    elif x=='north':
49
        return 3
50
    else:
51
        return 4
52

53
## 对护理来源进行编码
54
def care(x):
55
    if x=='Governament Hospital':
56
        return 1
57
    if x=='Never Counsulted':
58
        return 2
59
    if x=='Private Hospital' or x==' ':
60
        return 3
61
    if x=='clinic':
62
        return 4
63

64
mydata['性别']=mydata['性别'].apply(gender)
65
mydata['区域']=mydata['区域'].apply(district)
66
mydata['护理来源']=mydata['护理来源'].apply(care)
67

68
mytest['性别']=mytest['性别'].apply(gender)
69
mytest['区域']=mytest['区域'].apply(district)
70
mytest['护理来源']=mytest['护理来源'].apply(care)
71

72

73
## 对于数值型变量的缺失值用中位数进行填充
74
feature1 = ['最低血压','腰围','最高血压','体重指数','肥胖腰围','身高','体重','好胆固醇','总胆固醇','坏胆固醇']
75
for i in feature1:
76
    mydata[i] = mydata[i].fillna(mydata[i].median())
77
    mytest[i] = mytest[i].fillna(mytest[i].median())
78
## 对于分类型变量的缺失值用众数进行填充
79
feature2 = ['收入','未婚','视力不佳','高血压','慢性疲劳','肝炎','教育','糖尿病','家族肝炎','体育活动']
80
for i in feature2:
81
    mydata[i] = mydata[i].fillna(mydata[i].mode()[0])
82
    mytest[i] = mytest[i].fillna(mytest[i].mode()[0])
83

84

85
cut = []
86
for i in range(len(mydata['ALF'])):
87
    if mydata['ALF'][i] == 0:
88
        r = random.random()
89
        if r > 0.1:
90
            cut.append(i)
91
mydata = mydata.drop(cut)
92

93

94

95
## 导入需要的库
96
import xgboost as xgb
97
import numpy as np
98
from sklearn import datasets
99
from sklearn.metrics import accuracy_score
100

101
X = mydata.drop(['ALF',"id"], axis=1)
102
# X = mydata.drop(['ALF'], axis=1)
103
y = mydata['ALF']
104
X_train, X_val, y_train, y_val = train_test_split(X, y, test_size=0.3, stratify=y)
105

106

107
import lightgbm as lgb
108

109
clf7 = lgb.LGBMClassifier(
110
    max_depth=3,
111
    n_estimators=2000,
112
    n_jobs=-1,
113
    verbose=-1,
114
    learning_rate=0.1,
115
)
116
clf7.fit(X_train, y_train, eval_set=[(X_val,y_val)], callbacks=[lgb.early_stopping(50)])
117
val_y_predict = clf7.predict(X_val)
118

119
roc_auc_score(y_val, val_y_predict) #使用sklearn进行比较正确率
120

121

122
mytest['ALF']=0
123
X_test = mytest.drop(['ALF',"id"], axis=1)
124
y_test= mytest['ALF']
125
val_preds = clf7.predict(X_test)
126
result_list = []
127
for i in range(len(val_preds)):
128
    userid = mytest['id'][i]
129
    ALF = val_preds[i]
130
    line=[userid,ALF]
131
    result_list.append(line)
132
print(result_list[:20])
133
df = pd.DataFrame(result_list, columns=['id','ALF'])
134
df.to_csv('0721-1329.csv', index=False)

使用欠采样、模拟voting（0.834）#

1
## 直接导入之前已经处理好的数据
2
import pandas as pd
3
import re
4
mydata=pd.read_csv('/home/mw/input/MLearn9130/训练营.csv')
5
mytest=pd.read_csv('/home/mw/input/MLearn9130/测试集.csv')
6
## 分离训练集和验证集
7
from sklearn.model_selection import train_test_split
8

9
## 原始列名列表
10
col_names=list(mydata.columns)
11
col=[]
12
for i in range(len(col_names)):
13
    if re.findall(r"\u2028(.+)",col_names[i])!=[]:
14
        col.append(re.findall(r"\u2028(.+)",col_names[i])[0])
15
    elif re.findall(r"\n(.+)",col_names[i])!=[]:
16
        col.append(re.findall(r"\n(.+)",col_names[i])[0])
17
    else:
18
        col.append(col_names[i])
19

20
col_names1=list(mytest.columns)
21
col1=[]
22
for i in range(len(col_names1)):
23
    if re.findall(r"\u2028(.+)",col_names1[i])!=[]:
24
        col1.append(re.findall(r"\u2028(.+)",col_names1[i])[0])
25
    elif re.findall(r"\n(.+)",col_names1[i])!=[]:
26
        col1.append(re.findall(r"\n(.+)",col_names1[i])[0])
27
    else:
28
        col1.append(col_names1[i])
29

30
## 修改dataframe列名
31
mydata.columns=col
32
mytest.columns=col1
33

34

35
## 对性别进行编码
36
def gender(x):
37
    if x=='M':
38
        return 0
39
    else:
40
        return 1
41

42
## 对区域进行编码
43
def district(x):
44
    if x=='east':
45
        return 1
46
    elif x=='south':
47
        return 2
48
    elif x=='north':
49
        return 3
50
    else:
51
        return 4
52

53
## 对护理来源进行编码
54
def care(x):
55
    if x=='Governament Hospital':
56
        return 1
57
    if x=='Never Counsulted':
58
        return 2
59
    if x=='Private Hospital' or x==' ':
60
        return 3
61
    if x=='clinic':
62
        return 4
63

64
mydata['性别']=mydata['性别'].apply(gender)
65
mydata['区域']=mydata['区域'].apply(district)
66
mydata['护理来源']=mydata['护理来源'].apply(care)
67

68
mytest['性别']=mytest['性别'].apply(gender)
69
mytest['区域']=mytest['区域'].apply(district)
70
mytest['护理来源']=mytest['护理来源'].apply(care)
71

72

73
## 对于数值型变量的缺失值用中位数进行填充
74
feature1 = ['最低血压','腰围','最高血压','体重指数','肥胖腰围','身高','体重','好胆固醇','总胆固醇','坏胆固醇']
75
for i in feature1:
76
    mydata[i] = mydata[i].fillna(mydata[i].median())
77
    mytest[i] = mytest[i].fillna(mytest[i].median())
78
## 对于分类型变量的缺失值用众数进行填充
79
feature2 = ['收入','未婚','视力不佳','高血压','慢性疲劳','肝炎','教育','糖尿病','家族肝炎','体育活动']
80
for i in feature2:
81
    mydata[i] = mydata[i].fillna(mydata[i].mode()[0])
82
    mytest[i] = mytest[i].fillna(mytest[i].mode()[0])
83

84

85
import random
86
print("111", len(mydata[mydata['ALF'] == 0.0]))
87
print("111", len(mydata[mydata['ALF'] == 1.0]))
88
mydata_list = [[] for i in range(10)]
89
cnt = 0
90

91
for i in range(len(mydata)):
92
    if mydata['ALF'][i] == 0:
93
        mydata_list[cnt%10].append(mydata.iloc[i].tolist())
94
        cnt += 1
95
    else:
96
        for j in range(10):
97
            mydata_list[j].append(mydata.iloc[i].tolist())
98
for i in range(10):
99
    mydata_list[i] = pd.DataFrame(mydata_list[i], columns = list(mydata.columns))
100
    print(len(mydata_list[i]))
101

102
"""
103

104
111 3880
105
111 320
106
708
107
708
108
708
109
708
110
708
111
708
112
708
113
708
114
708
115
708
116
"""
117

118

119

120
## 导入需要的库
121
import xgboost as xgb
122
import numpy as np
123
import lightgbm as lgb
124
from sklearn.metrics import roc_auc_score
125
score = []
126
clf_list = []
127
for part_data in mydata_list:
128
    X = part_data.drop(['ALF',"id"], axis=1)
129
    y = part_data['ALF']
130
    X_train, X_val, y_train, y_val = train_test_split(X, y, test_size=0.3, stratify=y)
131

132
    clf = lgb.LGBMClassifier(
133
        max_depth=3,
134
        n_estimators=2000,
135
        n_jobs=-1,
136
        verbose=-1,
137
        learning_rate=0.15
138
    )
139
    clf.fit(X_train, y_train, eval_set=[(X_val,y_val)], callbacks=[lgb.early_stopping(20)])
140

141

142
    val_y_predict = clf.predict(X_val)
143

144
    this_score = roc_auc_score(y_val, val_y_predict)
145
    print(this_score)
146
    score.append(this_score) #使用sklearn进行比较正确率
147
    clf_list.append(clf)
148

149

150
score
151
"""
152
[0.7685630341880343,
153
 0.7553418803418803,
154
 0.8179754273504273,
155
 0.813167735042735,
156
 0.8202457264957264,
157
 0.7728365384615385,
158
 0.7799145299145299,
159
 0.8051549145299145,
160
 0.8050213675213675,
161
 0.8012820512820513]
162
"""
163

164
score_edit = [ i/sum(score) for i in score]
165
score_edit
166
"""
167
[0.0968024087063296,
168
 0.09513717178853173,
169
 0.10302602142941246,
170
 0.10242048073203143,
171
 0.10331197120317571,
172
 0.09734066710400163,
173
 0.09823215757514593,
174
 0.10141124623639637,
175
 0.10139442566146913,
176
 0.1009234495635061]
177
 """
178

179
mytest['ALF']=0
180
X_test = mytest.drop(['ALF',"id"], axis=1)
181
y_test= mytest['ALF']
182

183
val_preds = pd.DataFrame()
184
for i in range(10):
185
    val_preds['clf'+str(i)] = clf_list[i].predict(X_test)
186

187
val_preds['final'] = sum([ val_preds[i]*score_edit[int(i[-1])] for i in list(val_preds.columns)[:10]])
188

189
val_preds.to_csv('look.csv', index=False)
190
val_preds
191

192
"""
193
输出格式见下方
194
"""
195

196
result_list = []
197
val_preds_final = val_preds['final']
198
for i in range(len(val_preds)):
199
    userid = mytest['id'][i]
200
    # ALF = val_preds['clf3'][i]
201
    # ALF = 1.0 if val_preds_final[i] else 0.0
202
    ALF = 1.0 if val_preds_final[i]>0.52 else 0.0
203
    line=[userid,ALF]
204
    result_list.append(line)
205
print(result_list[:20])
206
df = pd.DataFrame(result_list, columns=['id','ALF'])
207
df.to_csv('0721-2029.csv', index=False)

Author Junyao Hu

Published Jul 20, 2022

Link https://junyaohu.github.io/blog/heywhale-summer-camp-ai/