基于tensorflow+CNN的新闻文本分类

2018年10月4日笔记

tensorflow是谷歌google的深度学习框架,tensor中文叫做张量,flow叫做流。 CNN是convolutional neural network的简称,中文叫做卷积神经网络。 文本分类是NLP(自然语言处理)的经典任务。

0.编程环境

操作系统:Win10 tensorflow版本:1.6 tensorboard版本:1.6 python版本:3.6

1.致谢声明

本文是作者学习《使用卷积神经网络以及循环神经网络进行中文文本分类》的成果,感激前辈; github链接:https://github.com/gaussic/text-classification-cnn-rnn

2.配置环境

使用循环神经网络模型要求有较高的机器配置,如果使用CPU版tensorflow会花费大量时间。 读者在有nvidia显卡的情况下,安装GPU版tensorflow会提高计算速度50倍。 安装教程链接:https://blog.csdn.net/qq_36556893/article/details/79433298 如果没有nvidia显卡,但有visa信用卡,请阅读我的另一篇文章《在谷歌云服务器上搭建深度学习平台》,链接:https://www.jianshu.com/p/893d622d1b5a

3.下载并解压数据集

数据集下载链接: https://pan.baidu.com/s/1oLZZF4AHT5X_bzNl2aF2aQ 提取码: 5sea 下载压缩文件cnews.zip完成后,选择解压到cnews。 文件夹结构如下图所示:

image.png

4.完整代码

代码文件需要放到和cnews文件夹同级目录

with open('./cnews/cnews.train.txt', encoding='utf8') as file:
    line_list = [k.strip() for k in file.readlines()]
    train_label_list = [k.split()[0] for k in line_list]
    train_content_list = [k.split(maxsplit=1)[1] for k in line_list]
with open('./cnews/cnews.vocab.txt', encoding='utf8') as file:
    vocabulary_list = [k.strip() for k in file.readlines()]
word2id_dict = dict(((b, a) for a, b in enumerate(vocabulary_list)))
content2vector = lambda content : [word2id_dict[word] for word in content if word in word2id_dict]
train_vector_list = [content2vector(content) for content in train_content_list ]
vocab_size = 5000  # 词汇表达小
embedding_dim = 64  # 词向量维度
seq_length = 600  # 序列长度
num_classes = 10  # 类别数
num_filters = 256  # 卷积核数目
kernel_size = 5  # 卷积核尺寸
hidden_dim = 128  # 全连接层神经元
dropout_keep_prob = 0.5  # dropout保留比例
learning_rate = 1e-3  # 学习率
batch_size = 64  # 每批训练大小
import tensorflow.contrib.keras as kr
train_X = kr.preprocessing.sequence.pad_sequences(train_vector_list, 600)
from sklearn.preprocessing import LabelEncoder
labelEncoder = LabelEncoder()
train_y = labelEncoder.fit_transform(train_label_list)
train_Y = kr.utils.to_categorical(train_y, num_classes=10)
import tensorflow as tf
tf.reset_default_graph()
X_holder = tf.placeholder(tf.int32, [None, seq_length])
Y_holder = tf.placeholder(tf.float32, [None, num_classes])

embedding = tf.get_variable('embedding', [vocab_size, embedding_dim])
embedding_inputs = tf.nn.embedding_lookup(embedding, X_holder)
conv = tf.layers.conv1d(embedding_inputs, num_filters, kernel_size)
max_pooling = tf.reduce_max(conv, reduction_indices=[1])
full_connect = tf.layers.dense(max_pooling, hidden_dim)
full_connect_dropout = tf.contrib.layers.dropout(full_connect, keep_prob=1)
full_connect_activate = tf.nn.relu(full_connect_dropout)
softmax_before = tf.layers.dense(full_connect_activate, num_classes)
predict_Y = tf.nn.softmax(softmax_before)
cross_entropy = tf.nn.softmax_cross_entropy_with_logits_v2(labels=Y_holder, logits=softmax_before)
loss = tf.reduce_mean(cross_entropy)
optimizer = tf.train.AdamOptimizer(learning_rate)
train = optimizer.minimize(loss)
isCorrect = tf.equal(tf.argmax(Y_holder, 1), tf.argmax(predict_Y, 1))
accuracy = tf.reduce_mean(tf.cast(isCorrect, tf.float32))

init = tf.global_variables_initializer()
session = tf.Session()
session.run(init)

import random
for i in range(3000):
    selected_index = random.sample(list(range(len(train_y))), k=64)
    batch_X = train_X[selected_index]
    batch_Y = train_Y[selected_index]
    session.run(train, {X_holder:batch_X, Y_holder:batch_Y})
    step = i + 1 
    if step % 100 == 0:
        selected_index = random.sample(list(range(len(train_y))), k=200)
        batch_X = train_X[selected_index]
        batch_Y = train_Y[selected_index]
        loss_value, accuracy_value = session.run([loss, accuracy], {X_holder:batch_X, Y_holder:batch_Y})
        print('step:%d loss:%.4f accuracy:%.4f' %(step, loss_value, accuracy_value))

上面一段代码的运行结果如下(只截取前十行):

step:100 loss:0.5491 accuracy:0.8650 step:200 loss:0.2495 accuracy:0.9200 step:300 loss:0.1928 accuracy:0.9450 step:400 loss:0.1123 accuracy:0.9700 step:500 loss:0.1183 accuracy:0.9800 step:600 loss:0.0946 accuracy:0.9800 step:700 loss:0.1316 accuracy:0.9600 step:800 loss:0.1455 accuracy:0.9650 step:900 loss:0.1226 accuracy:0.9600 step:1000 loss:0.0686 accuracy:0.9800

5.数据准备

with open('./cnews/cnews.train.txt', encoding='utf8') as file:
    line_list = [k.strip() for k in file.readlines()]
    train_label_list = [k.split()[0] for k in line_list]
    train_content_list = [k.split(maxsplit=1)[1] for k in line_list]
with open('./cnews/cnews.vocab.txt', encoding='utf8') as file:
    vocabulary_list = [k.strip() for k in file.readlines()]
word2id_dict = dict(((b, a) for a, b in enumerate(vocabulary_list)))
content2vector = lambda content : [word2id_dict[word] for word in content if word in word2id_dict]
train_vector_list = [content2vector(content) for content in train_content_list ]
vocab_size = 5000  # 词汇表达小
embedding_dim = 64  # 词向量维度
seq_length = 600  # 序列长度
num_classes = 10  # 类别数
num_filters = 256  # 卷积核数目
kernel_size = 5  # 卷积核尺寸
hidden_dim = 128  # 全连接层神经元
dropout_keep_prob = 0.5  # dropout保留比例
learning_rate = 1e-3  # 学习率
batch_size = 64  # 每批训练大小
import tensorflow.contrib.keras as kr
train_X = kr.preprocessing.sequence.pad_sequences(train_vector_list, 600)
from sklearn.preprocessing import LabelEncoder
labelEncoder = LabelEncoder()
train_y = labelEncoder.fit_transform(train_label_list)
train_Y = kr.utils.to_categorical(train_y, num_classes=10)
import tensorflow as tf
tf.reset_default_graph()
X_holder = tf.placeholder(tf.int32, [None, seq_length])
Y_holder = tf.placeholder(tf.float32, [None, num_classes])

6.搭建神经网络

embedding = tf.get_variable('embedding', [vocab_size, embedding_dim])
embedding_inputs = tf.nn.embedding_lookup(embedding, X_holder)
conv = tf.layers.conv1d(embedding_inputs, num_filters, kernel_size)
max_pooling = tf.reduce_max(conv, reduction_indices=[1])
full_connect = tf.layers.dense(max_pooling, hidden_dim)
full_connect_dropout = tf.contrib.layers.dropout(full_connect, keep_prob=1)
full_connect_activate = tf.nn.relu(full_connect_dropout)
softmax_before = tf.layers.dense(full_connect_activate, num_classes)
predict_Y = tf.nn.softmax(softmax_before)
cross_entropy = tf.nn.softmax_cross_entropy_with_logits_v2(labels=Y_holder, logits=softmax_before)
loss = tf.reduce_mean(cross_entropy)
optimizer = tf.train.AdamOptimizer(learning_rate)
train = optimizer.minimize(loss)
isCorrect = tf.equal(tf.argmax(Y_holder, 1), tf.argmax(predict_Y, 1))
accuracy = tf.reduce_mean(tf.cast(isCorrect, tf.float32))

7.参数初始化

init = tf.global_variables_initializer()
session = tf.Session()
session.run(init)

8.模型训练

import random
for i in range(3000):
    selected_index = random.sample(list(range(len(train_y))), k=64)
    batch_X = train_X[selected_index]
    batch_Y = train_Y[selected_index]
    session.run(train, {X_holder:batch_X, Y_holder:batch_Y})
    step = i + 1 
    if step % 100 == 0:
        selected_index = random.sample(list(range(len(train_y))), k=200)
        batch_X = train_X[selected_index]
        batch_Y = train_Y[selected_index]
        loss_value, accuracy_value = session.run([loss, accuracy], {X_holder:batch_X, Y_holder:batch_Y})
        print('step:%d loss:%.4f accuracy:%.4f' %(step, loss_value, accuracy_value))

本文参与腾讯云自媒体分享计划,欢迎正在阅读的你也加入,一起分享。

发表于

我来说两句

0 条评论
登录 后参与评论

相关文章

来自专栏图像识别与深度学习

2018-06-30 TensorFlow实战:SoftMax手写体MNIST识别(Python完整源码)

1484
来自专栏深度学习思考者

Keras:基于Theano和TensorFlow的深度学习库之中文文档

Keras官网:http://keras.io/ Github项目:https://github.com/fchollet/keras 中文文档主页:http:...

22410
来自专栏王相军的专栏

gcForest 集成学习方法的 Python 实现

前言 看完 gcForest 这篇 paper 有一段时间了,但是一直没有去网上搜集相关的实现代码,去把它调试跑通,并将之应用到实际的项目中。这两天终于抽空做了...

3.5K9
来自专栏云时之间

深度学习的fine-tuning过程

在深度学习过程中,获取数据集后,在搭建自己的网络之前需要进行的是微调,通过别人现有的网络观察自己数据的实验结果,并在此基础上,初步确定自己网络的大体结构,其中微...

3706
来自专栏杨熹的专栏

TensorFlow-2: 用 CNN 识别数字

---- 本文结构: CNN 建立模型 code ---- 昨天只是用了简单的 softmax 做数字识别,准确率为 92%,这个太低了,今天用 CNN 来提高...

3605
来自专栏PaddlePaddle

【图像分类】如何转化模型文件

场景文字识别 图像相比文字能够提供更加生动、容易理解及更具艺术感的信息,是人们转递与交换信息的重要来源。图像分类是根据图像的语义信息对不同类别图像进行区分,是计...

3595
来自专栏杨熹的专栏

TensorFlow -2: 用 CNN 识别数字

昨天只是用了简单的 softmax 做数字识别,准确率为 92%,这个太低了,今天用 CNN 来提高一下准确率。关于 CNN,可以看这篇:图解何为CNN简单看一...

1.6K0
来自专栏AI研习社

Github 代码实践:Pytorch 实现的语义分割器

使用Detectron预训练权重输出 *e2e_mask_rcnn-R-101-FPN_2x* 的示例

2612
来自专栏数据科学与人工智能

Python玩机器学习简易教程

本文介绍利用Python和Python的机器学习库scikit-learn完成一个端到端的机器学习项目。 俗话说,“师傅领进门,修行在个人”。本文就是扮演领进门...

4457
来自专栏ATYUN订阅号

【教程】利用Tensorflow目标检测API确定图像中目标的位置

深度学习提供了另一种解决“Wally在哪儿”(美国漫画)问题的方法。与传统的图像处理计算机视觉方法不同的是,它只使用了少量的标记出Wally位置的示例。 在我的...

7026

扫码关注云+社区

领取腾讯云代金券