文本相似在问答系统中有很重要的应用,如基于知识的问答系统(Knowledge-based QA),基于文档的问答系统(Documen-based QA),以及基于FAQ的问答系统(Community-QA)等。像 对于问题的内容,需要进行相似度匹配,从而选择出与问题最接近,同时最合理的答案。本节介绍 基于bert的余弦距离计算相似度。
学习bert可以看这里:https://cloud.tencent.com/developer/article/1590283
训练/预测:
# 绘图案例 an example of matplotlib
import numpy as np
import matplotlib.pyplot as plt
from scipy.special import jn
from IPython.display import display, clear_output
import time
from sklearn.model_selection import KFold, train_test_split, GridSearchCV
'''
x = np.linspace(0,5)
f, ax = plt.subplots()
ax.set_title("Bessel functions")
for n in range(1,10):
time.sleep(1)
ax.plot(x, jn(x,n))
clear_output(wait=True)
display(f)
# close the figure at the end, so we don't get a duplicate
# of the last plot
plt.close()
'''
from keras.layers import *
from bert4keras.backend import keras, set_gelu