前言
近期,技术圈被一个名为Pykan的项目彻底点燃,其在GitHub上的关注度火箭般攀升,逼近万星大关,引得无数开发者侧目
像什么xx之心,量子x, 新x元等top公众号都写了相关文章
有人感慨前苏联数理水平之高,有人惊叹作者脑洞大开,更有人称之为未来已来
当然也不乏质疑的声音
本项目中小编将基于官方的示例进行一些简单测试
旨在剥开层层热度的外衣,与大家一窥它的真实面貌与潜力
注意
尽量使用核数较多的资源,例如我目前使用的4核16g 当前镜像为气象分析3.9
点击以下链接可在线运行Fork查看完整程序 🔜🔜若没有成功加载可视化图,点击运行可以查看 ps:隐藏代码在【代码已被隐藏】所在行,点击所在行,可以看到该行的最右角,会出现个三角形,点击查看即可
【前沿】Pykan风靡GitHub,近万收藏的含金量到底如何
其名为KAN
Kolmogorov-Arnold Networks (KANs) 是一类受到数学家 Kolmogorov 和 Arnold 的工作启发的神经网络结构,它们为多层感知器(MLPs)提供了一种有吸引力的替代方案。KANs 的设计基于 Kolmogorov-Arnold 表示定理,这与 MLPs 基于的通用近似定理形成对比。一个显著的区别在于 KANs 将激活函数置于网络的边(连接)上,而传统的 MLPs 则将激活函数应用于节点上。这一结构上的变化使得 KANs 在模型精度和可解释性方面有时能够显著优于 MLPs。
以上为github介绍机翻
在github界面介绍的最后,作者还写了一句话
Practice is the only criterion for testing understanding (实践是检验真理的唯一标准).
环境搭建
pykan需要matplotlib==3.6.2 numpy==1.24.4 scikit_learn==1.1.3 setuptools==65.5.0 sympy==1.11.1 torch==2.2.2 tqdm==4.66.2 当前镜像库比较齐全,我们只需要torch和pykan即可
In [5]:
pip install pykan -i https://pypi.mirrors.ustc.edu.cn/simple/
In [4]:
pip install --index-url https://pypi.mirrors.ustc.edu.cn/simple torch==2.2.2 --user
回归问题
我们首先将该问题视为回归问题(输出维数 = 1,MSE 损失)
建立一个双月数据集,官方示例是一千样本,我改为10000样本测试
In [1]:
from kan import KAN
import matplotlib.pyplot as plt
from sklearn.datasets import make_moons
import torch
import numpy as np
dataset = {}
train_input, train_label = make_moons(n_samples=10000, shuffle=True, noise=0.1, random_state=None)
test_input, test_label = make_moons(n_samples=10000, shuffle=True, noise=0.1, random_state=None)
dataset['train_input'] = torch.from_numpy(train_input)
dataset['test_input'] = torch.from_numpy(test_input)
dataset['train_label'] = torch.from_numpy(train_label[:,None])
dataset['test_label'] = torch.from_numpy(test_label[:,None])
X = dataset['train_input']
y = dataset['train_label']
plt.scatter(X[:,0], X[:,1], c=y[:,0])
支持向量机
In [10]:
from sklearn import svm
from sklearn.model_selection import train_test_split
from sklearn.metrics import classification_report
# 将数据集分为训练集和测试集
X_train, X_test, y_train, y_test = train_test_split(train_input, train_label, test_size=0.2, random_state=42)
# 初始化SVM模型,这里我们使用RBF核函数
clf = svm.SVC(kernel='rbf', C=1, gamma='scale')
# 训练模型
clf.fit(X_train, y_train)
# 预测并评估
y_pred = clf.predict(X_test)
print(classification_report(y_test, y_pred))
precision recall f1-score support
0 1.00 0.99 1.00 107
1 0.99 1.00 0.99 93
accuracy 0.99 200
macro avg 0.99 1.00 0.99 200
weighted avg 1.00 0.99 1.00 200
KAN
model = KAN(width=[2,1], grid=3, k=3)
def train_acc():
return torch.mean((torch.round(model(dataset['train_input'])[:,0]) == dataset['train_label'][:,0]).float())
def test_acc():
return torch.mean((torch.round(model(dataset['test_input'])[:,0]) == dataset['test_label'][:,0]).float())
results = model.train(dataset, opt="LBFGS", steps=20, metrics=(train_acc, test_acc));
results['train_acc'][-1], results['test_acc'][-1]
自动符号回归
In [5]:
lib = ['x','x^2','x^3','x^4','exp','log','sqrt','tanh','sin','tan','abs']
model.auto_symbolic(lib=lib)
formula = model.symbolic_formula()[0][0]
formula
fixing (0,0,0) with sin, r2=0.9731504496053569
fixing (0,1,0) with tan, r2=0.978959743323539−
0.39sin(3.1𝑥1−1.53)−0.82tan(0.92𝑥2−3.39)+0.48
这个公式有多准确?
In [6]:
#
def acc(formula, X, y):
batch = X.shape[0]
correct = 0
for i in range(batch):
correct += np.round(np.array(formula.subs('x_1', X[i,0]).subs('x_2', X[i,1])).astype(np.float64)) == y[i,0]
return correct/batch
print('train acc of the formula:', acc(formula, dataset['train_input'], dataset['train_label']))
print('test acc of the formula:', acc(formula, dataset['test_input'], dataset['test_label']))
train acc of the formula: tensor(0.9987)
test acc of the formula: tensor(0.9981)
分类方程
(输出维数 = 2,交叉熵损失)。
In [11]:
from kan import KAN
import matplotlib.pyplot as plt
from sklearn.datasets import make_moons
import torch
import numpy as np
dataset = {}
train_input, train_label = make_moons(n_samples=10000, shuffle=True, noise=0.1, random_state=None)
test_input, test_label = make_moons(n_samples=10000, shuffle=True, noise=0.1, random_state=None)
dataset['train_input'] = torch.from_numpy(train_input)
dataset['test_input'] = torch.from_numpy(test_input)
dataset['train_label'] = torch.from_numpy(train_label)
dataset['test_label'] = torch.from_numpy(test_label)
X = dataset['train_input']
y = dataset['train_label']
plt.scatter(X[:,0], X[:,1], c=y[:])
<matplotlib.collections.PathCollection at 0x7f80a3933b20>
向量机
In [12]:
from sklearn import svm
from sklearn.preprocessing import StandardScaler
from sklearn.metrics import accuracy_score
# 数据预处理:对输入数据进行标准化
scaler = StandardScaler()
train_input_scaled = scaler.fit_transform(train_input)
test_input_scaled = scaler.transform(test_input)
# 初始化SVM模型,选择RBF核函数
svm_model = svm.SVC(kernel='rbf')
# 训练模型
svm_model.fit(train_input_scaled, train_label.ravel())
# 预测测试集
predictions = svm_model.predict(test_input_scaled)
# 计算并打印准确率
accuracy = accuracy_score(test_label, predictions)
print(f"SVM with RBF Kernel Accuracy: {accuracy}")
SVM with RBF Kernel Accuracy: 0.9993
简易神经网络
In [13]:
import torch
import torch.nn as nn
import torch.optim as optim
# 将numpy数组转换为torch张量
train_input_tensor = torch.FloatTensor(train_input)
train_label_tensor = torch.LongTensor(train_label)
test_input_tensor = torch.FloatTensor(test_input)
test_label_tensor = torch.LongTensor(test_label)
# 定义神经网络结构
class SimpleClassifier(nn.Module):
def __init__(self, input_dim, hidden_dim, output_dim):
super(SimpleClassifier, self).__init__()
self.layer1 = nn.Linear(input_dim, hidden_dim)
self.layer2 = nn.Linear(hidden_dim, output_dim)
def forward(self, x):
x = torch.relu(self.layer1(x))
x = self.layer2(x)
return x
# 实例化模型、损失函数和优化器
input_dim = train_input.shape[1]
hidden_dim = 10
output_dim = 2 # 假设是二分类问题
model1 = SimpleClassifier(input_dim, hidden_dim, output_dim)
criterion = nn.CrossEntropyLoss()
optimizer = optim.Adam(model1.parameters(), lr=0.01)
# 训练神经网络
num_epochs = 100
for epoch in range(num_epochs):
# 前向传播
outputs = model1(train_input_tensor)
loss = criterion(outputs, train_label_tensor)
# 反向传播和优化
optimizer.zero_grad()
loss.backward()
optimizer.step()
if (epoch+1) % 10 == 0:
print(f'Epoch [{epoch+1}/{num_epochs}], Loss: {loss.item()}')
# 测试集评估
with torch.no_grad():
outputs = model1(test_input_tensor)
_, predicted = torch.max(outputs.data, 1)
accuracy = (predicted == test_label_tensor).sum().item() / len(test_label_tensor)
print(f"Neural Network Accuracy: {accuracy}")
Epoch [10/100], Loss: 0.6714565753936768
Epoch [20/100], Loss: 0.519657552242279
Epoch [30/100], Loss: 0.4016408622264862
Epoch [40/100], Loss: 0.3351301848888397
Epoch [50/100], Loss: 0.30125096440315247
Epoch [60/100], Loss: 0.2788920998573303
Epoch [70/100], Loss: 0.26166898012161255
Epoch [80/100], Loss: 0.24820241332054138
Epoch [90/100], Loss: 0.23750551044940948
Epoch [100/100], Loss: 0.22848664224147797
Neural Network Accuracy: 0.9034
KAN训练
In [14]:
model = KAN(width=[2,2], grid=3, k=3)
def train_acc():
return torch.mean((torch.argmax(model(dataset['train_input']), dim=1) == dataset['train_label']).float())
def test_acc():
return torch.mean((torch.argmax(model(dataset['test_input']), dim=1) == dataset['test_label']).float())
results = model.train(dataset, opt="LBFGS", steps=20, metrics=(train_acc, test_acc), loss_fn=torch.nn.CrossEntropyLoss());
自动符号回归
In [15]:
lib = ['x','x^2','x^3','x^4','exp','log','sqrt','tanh','sin','abs']
model.auto_symbolic(lib=lib)
fixing (0,0,0) with x^3, r2=0.9097426990326932
fixing (0,0,1) with tanh, r2=0.9344546561832844
fixing (0,1,0) with tanh, r2=0.9959303072149012
fixing (0,1,1) with exp, r2=0.975737216980336
formula1, formula2 = model.symbolic_formula()[0]
formula1
In [17]:
formula2
301.53
这个公式有多准确?
In [18]:
def acc(formula1, formula2, X, y):
batch = X.shape[0]
correct = 0
for i in range(batch):
logit1 = np.array(formula1.subs('x_1', X[i,0]).subs('x_2', X[i,1])).astype(np.float64)
logit2 = np.array(formula2.subs('x_1', X[i,0]).subs('x_2', X[i,1])).astype(np.float64)
correct += (logit2 > logit1) == y[i]
return correct/batch
print('train acc of the formula:', acc(formula1, formula2, dataset['train_input'], dataset['train_label']))
print('test acc of the formula:', acc(formula1, formula2, dataset['test_input'], dataset['test_label']))
train acc of the formula: tensor(0.8790)
test acc of the formula: tensor(0.8790)
小结
可以下一个初步的使用感受:
1. pykan看得出是一个学者制作的python库,
它更加倾向于探索数学物理定律,希望这个库能够为我所用,在气象领域有所建树 2. 正如大家所说,它的训练过程是较慢的,使用过程是非常吃资源的,但是它的版本目前为0.0.5,未来可期 3. 有没有到能颠覆业界的水平,以我浅薄的认知无法下判断,反正业界一三五要被颠覆,二四六被震惊 4. 在分类问题上准确率虽然被向量机吊打,但是能给出公式,这算是在可解释性上迈出一步了(当然准确率也可能是数据集的问题) 5. 限于个人水平,在此仅仅做简单测试,如有错漏,还请指正