权力的游戏使用Tensorflow中的LSTM和Recurrent单元生成剧集脚本
权力的游戏使用Tensorflow中的LSTM和Recurrent单元生成剧集脚本
代码医生工作室
发布于 2019-08-09 20:08:37
发布于 2019-08-09 20:08:37
文章被收录于专栏：相约机器人
来源 | Medium
编辑 | 代码医生团队
权力的游戏第8季确实令人失望，似乎导演无法学习前几季的模式，但担心的是，LSTM从不会错过任何模式，并会帮助提供一个更好的脚本，一个AI生成的脚本。
在深入研究模型的代码和脚本以及它的训练之前，将简要介绍一下LSTM（长期短期记忆）细胞以及它们如何有用。
LSTM概念
LSTM网络是一种递归神经网络，它具有LSTM单元块代替我们的常规神经网络层。这些单元有不同的段，称为输入门，忘记门和输出门，如下图所示 -
LSTM门
下面给出的图像显示了门如何操作以及每个门所涉及的数学方程，这使得它们的功能变得重要且可执行。
LSTM门学习过程
实施和准则
将建立一个在Anna Karenina上训练的角色智能RNN，一旦训练完成并且对前者进行测试，将由权力的游戏第3季和第4季的联合脚本取代。之后它将能够根据季节脚本中的文本生成新文本。
import time
from collections import namedtuple
import numpy as np
import tensorflow as tf首先，加载文本文件并将其更改为整数，以便网络继续工作。在这里，将构建一些字典来将字符转换为整数和从整数转换。将字符编码为整数使得它更易于用作网络中的输入以进行训练。
with open('anna.txt', 'r') as f:
text=f.read()
vocab = sorted(set(text))
vocab_to_int = {c: i for i, c in enumerate(vocab)}
int_to_vocab = dict(enumerate(vocab))
encoded = np.array([vocab_to_int[c] for c in text], dtype=np.int32)在完成这一步骤后，继续为训练过程生成小批量产品，将编码如下
批量训练
'''Create a generator that returns batches of size batch_size x n_steps from arr '''
def get_batches(arr, batch_size, n_steps):
'''
arr: Array you want to make batches from
batch_size: Batch size, the number of sequences per batch
n_steps: Number of sequence steps per batch
'''
# Get the number of characters per batch
chars_per_batch = batch_size * n_steps
# Number of batches to be made
n_batches = len(arr)//chars_per_batch
# Make full batches with enough characters
arr = arr[:n_batches * chars_per_batch]
# Reshape into batch_size rows
arr = arr.reshape((batch_size, -1))
for n in range(0, arr.shape[1], n_steps):
x = arr[:, n:n+n_steps]
y_temp = arr[:, n+1:n+n_steps+1]
y = np.zeros(x.shape, dtype=x.dtype)
y[:,:y_temp.shape[1]] = y_temp
yield x, y获取用于生成迷你批次的批次功能
注意
希望批次是具有一些所需数量的序列步骤的多个序列。
在定义了一个将生成小批量的函数之后，现在将生成批量大小为10和50个序列步骤的数据集。
batches = get_batches(encoded, 10, 50)
x, y = next(batches)现在是开始构建网络的部分，为了简化流程，将其分解为多个部分，以便设计和理解每个部分更加舒适。然后稍后可以将它们组合到整个网络中。
创建输入
将首先为训练数据和目标创建输入占位符以及用于丢失层的占位符。
''' Define placeholders for inputs, targets, and dropout '''
def build_inputs(batch_size, num_steps):
'''
batch_size: Batch size, number of sequences per batch
num_steps: Number of sequence steps in a batch
'''
# Declare placeholders to feed into the graph
inputs = tf.placeholder(tf.int32, [batch_size, num_steps], name='inputs')
targets = tf.placeholder(tf.int32, [batch_size, num_steps], name='targets')
# Probability placeholder for drop out layers
keep_prob = tf.placeholder(tf.float32, name='keep_prob')
return inputs, targets, keep_probLSTM Cell
现在将使用RNN作为Recurrent cell功能的构建块在隐藏层中创建LSTM单元。
def build_lstm(lstm_size, num_layers, batch_size, keep_prob):
'''
keep_prob: Scalar tensor (tf.placeholder) for the dropout keep probability
lstm_size: Size of the hidden layers in the LSTM cells
num_layers: Number of LSTM layers
batch_size: Batch size
'''
def build_cell(lstm_size, keep_prob):
lstm = tf.contrib.rnn.BasicLSTMCell(lstm_size)
drop = tf.contrib.rnn.DropoutWrapper(lstm, output_keep_prob=keep_prob)
return drop
# Stacking up multiple LSTM layers
cell = tf.contrib.rnn.MultiRNNCell([build_cell(lstm_size, keep_prob) for _ in range(num_layers)])
initial_state = cell.zero_state(batch_size, tf.float32)
return cell, initial_state输出
差不多完成了，只需要将RNN单元的输出连接到具有softmax输出的完全连接层。softmax输出提供了一个概率分布，可以利用它来预测下面的字符。
### Build a softmax layer, return the softmax output and logits
def build_output(lstm_output, in_size, out_size):
'''
x: Input tensor
in_size: Size of the input tensor, for example, size of the LSTM cells
out_size: Size of this softmax layer
'''
# Reshape output
# shape = batch_size*num_steps rows by lstm_size columns
seq_output = tf.concat(lstm_output, axis=1)
x = tf.reshape(seq_output, [-1, in_size])
# Connecting the RNN outputs to a softmax layer
with tf.variable_scope('softmax'):
softmax_w = tf.Variable(tf.truncated_normal((in_size, out_size), stddev=0.1))
softmax_b = tf.Variable(tf.zeros(out_size))
logits = tf.matmul(x, softmax_w) + softmax_b
# Softmax to get the probabilities for predicted characters
out = tf.nn.softmax(logits, name='predictions')
return out, logits损失和优化
使用Cross-Entropy Loss和Adam Optimizer进行学习步骤。
def build_loss(logits, targets, lstm_size, num_classes):
y_one_hot = tf.one_hot(targets, num_classes)
y_reshaped = tf.reshape(y_one_hot, logits.get_shape())
loss = tf.nn.softmax_cross_entropy_with_logits(logits=logits, labels=y_reshaped)
loss = tf.reduce_mean(loss)
return loss
def build_optimizer(loss, learning_rate, grad_clip):
tvars = tf.trainable_variables()
grads, _ = tf.clip_by_global_norm(tf.gradients(loss, tvars), grad_clip)
train_op = tf.train.AdamOptimizer(learning_rate)
optimizer = train_op.apply_gradients(zip(grads, tvars))
return optimizer在对所有上述材料进行编码之后，现在是将所有这些材料合并为最终作品的时间
class CharRNN:
def __init__(self, num_classes, batch_size=64, num_steps=50,
lstm_size=128, num_layers=2, learning_rate=0.001,
grad_clip=5, sampling=False):
if sampling == True:
batch_size, num_steps = 1, 1
else:
batch_size, num_steps = batch_size, num_steps
tf.reset_default_graph()
# Build the input placeholder tensors
self.inputs, self.targets, self.keep_prob = build_inputs(batch_size, num_steps)
# Build the LSTM cell
cell, self.initial_state = build_lstm(lstm_size, num_layers, batch_size, self.keep_prob)
x_one_hot = tf.one_hot(self.inputs, num_classes)
outputs, state = tf.nn.dynamic_rnn(cell, x_one_hot, initial_state=self.initial_state)
self.final_state = state
self.prediction, self.logits = build_output(outputs, lstm_size, num_classes)
self.loss = build_loss(self.logits, self.targets, lstm_size, num_classes)
self.optimizer = build_optimizer(self.loss, learning_rate, grad_clip)训练网络
这是一个常规训练代码，在运行优化程序时将输入和目标提供给网络。收到小批量的终端LSTM位置。接下来，将该位置传递回网络，以便下一批可以保持早期批次的位置。
epochs = 20
# Print losses every N interations
print_every_n = 50
# Save every N iterations
save_every_n = 200
model = CharRNN(len(vocab), batch_size=batch_size, num_steps=num_steps,
lstm_size=lstm_size, num_layers=num_layers,
learning_rate=learning_rate)
saver = tf.train.Saver(max_to_keep=100)
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
# Use the line below to load a checkpoint and resume training
#saver.restore(sess, 'checkpoints/______.ckpt')
counter = 0
for e in range(epochs):
# Train network
new_state = sess.run(model.initial_state)
loss = 0
for x, y in get_batches(encoded, batch_size, num_steps):
counter += 1
start = time.time()
feed = {model.inputs: x,
model.targets: y,
model.keep_prob: keep_prob,
model.initial_state: new_state}
batch_loss, new_state, _ = sess.run([model.loss,
model.final_state,
model.optimizer],
feed_dict=feed)
if (counter % print_every_n == 0):
end = time.time()
print('Epoch: {}/{}... '.format(e+1, epochs),
'Training Step: {}... '.format(counter),
'Training loss: {:.4f}... '.format(batch_loss),
'{:.4f} sec/batch'.format((end-start)))
if (counter % save_every_n == 0):
saver.save(sess, "checkpoints/i{}_l{}.ckpt".format(counter, lstm_size))
saver.save(sess, "checkpoints/i{}_l{}.ckpt".format(counter, lstm_size))结果
checkpoint = tf.train.latest_checkpoint('checkpoints')
samp = sample(checkpoint, 10000, lstm_size, len(vocab), prime="Far")
print(samp)
此处使用的示例函数是更大代码的一部分，可以在存储库中找到它以及整个笔记本和代码片段来执行自己的代码。
https://github.com/Ujwal2910/LSTM_Book_Genearator?source=post_page---------------------------
权力的游戏NewScript时间
让LSTM嚎叫Dracrys
现在已经学会了如何使用LSTM生成新书，只需下载每集的字幕并将它们拼凑在一起形成一个季节。
完成后，使用自己的剧集/剧集/季节/季节文本文件替换存储库中的anna.text。
训练一个赛季并查看结果，然后继续增加更多赛季，以进一步优化数据集和学习过程。
本文参与 腾讯云自媒体同步曝光计划，分享自微信公众号。
原始发表：2019-08-08，如有侵权请联系 cloudcommunity@tencent.com 删除
评论
登录后参与评论
推荐阅读
腾讯云开发者
