python - 将代码从二元分类器逻辑回归修改为多类“一对多”逻辑回归
问题描述
我是机器学习的新手并尝试练习不同的算法,目前我使用逻辑回归对从 sklearn 生成的随机数据集进行分类。现在这是一个二元分类器,但是我想使用多类逻辑回归“one vs all”方法(稍后进行比较)。
下面是我尝试为二进制分类实现的代码:
import numpy as np
import matplotlib.pyplot as plt
import sklearn
import random
from sklearn import datasets
from sklearn.preprocessing import StandardScaler
from sklearn.datasets import make_blobs
X, t = make_blobs(n_samples=[400, 800, 400], centers=[[0,0], [1,2], [2,3]],
n_features=2, random_state=2019)
indices = np.arange(X.shape[0])
random.seed(2020)
random.shuffle(indices)
indices[:10]
X_train = X[indices[:800], :]
X_val = X[indices[800:1200], :]
X_test = X[indices[1200:], :]
t_train = t[indices[:800]]
t_val = t[indices[800:1200]]
t_test = t[indices[1200:]]
t2_train = t_train == 1
t2_train = t2_train.astype('int')
t2_val = (t_val == 1).astype('int')
t2_test = (t_test == 1).astype('int')
def add_bias(X):
# Put bias in position 0
sh = X.shape
if len(sh) == 1:
#X is a vector
return np.concatenate([np.array([1]), X])
else:
# X is a matrix
m = sh[0]
bias = np.ones((m, 1)) # Makes a m*1 matrix of 1-s
return np.concatenate([bias, X], axis = 1)
class NumpyClassifier():
# Common methods to all numpy classifiers --- if any
def accuracy(self, X_val, t_val, **kwargs):
pred = self.predict(X_val, **kwargs)
if len(pred.shape) > 1:
pred = pred[:, 0]
return sum(pred==t_val)/len(pred)
# code for Logistic Regression
def logistic(x):
return 1/(1+np.exp(-x))
class NumpyLogReg(NumpyClassifier):
def fit(self, X_train, t_train, gamma = 0.1, epochs=10):
# X_train is a Nxm matrix, N data points, m features
# t_train are the targets values for training data
(k, m) = X_train.shape
X_train = add_bias(X_train)
self.theta = theta = np.zeros(m+1)
for e in range(epochs):
theta -= gamma / k * X_train.T @ (self.forward(X_train) - t_train)
def forward(self, X_val):
return logistic(X_val @ self.theta)
def score(self, X_val):
z = add_bias(X_val)
score = self.forward(z)
return score
def predict(self, X_val, threshold=0.5):
z = add_bias(X_val)
score = self.forward(z)
# score = z @ self.theta
return (score>threshold).astype('int')
lr_cl = NumpyLogReg()
lr_cl.fit(X_train, t_train)
lr_cl.predict(X_val)
lr_cl.accuracy(X_val, t_val)
for e in [1, 2, 5, 10, 50, 100, 1000, 10000, 100000, 1000000]:
lr_cl = NumpyLogReg()
lr_cl.fit(X_train, t_train, epochs=e, gamma=0.00001)
print("{:10} {:7.3f}".format(e, lr_cl.accuracy(X_val, t_val)))
我需要关于如何将代码修改为多类“one vs all”/“one vs rest”Logistic Regression 的建议/提示。我不想直接使用 sklearn 导入的逻辑回归算法,而是像这样从头开始。
任何建议都非常感谢,在此先感谢您。
解决方案
我认为NumpyLogReg
二进制分类效果很好。使用(ovr) 技术对同一类进行multi-class
分类。One-Vs-Rest
假设数据集有 3 个类A, B, C
- 调用类标签
A
为 +ve class &B, C
as -ve class 的二元分类模型并记下概率分数 B
通过考虑as +ve &A, C
as -ve 和C
as +ve & as -ve重复相同的A, B
操作。记下各自的概率分数。- 基本上,如果有
n
类,就会有n
二元分类器模型,即,fitting one classifier per class
- 通过仔细检查每个类的分类器(即通过分析概率值),您可以实现
multi-class
分类并且模型将具有高度的可解释性。
请参阅本指南以获得更详细的说明
推荐阅读
- node.js - 如何从 app.js 导出函数并从另一个 js 文件导入?
- python - 在运行循环时获取线程内变量的值(python)
- powershell - Jenkins 参数与 powershell
- wpf - 没有 GDI+ 有没有办法进行图像阈值处理?
- xslt - 嵌套条件循环(一对多)
- xml - 如何在 VLC 的播放列表窗口中显示嵌套条目?
- database - YugaByte DB 中的 RHEL 环境
- java - 如何在拖动鼠标时平滑填充椭圆
- javascript - 使用javascript在html脚本库中对sqlite数据库执行查询
- http - 获取 https://
/translate/2327496366232: x509: 证书由未知机构签署”