Python实战——ESIM 模型搭建（keras版）

莫斯

发布于 2020-09-10 10:41:55

1.1K0

发布于 2020-09-10 10:41:55

文章被收录于专栏：备份

ESIM 原理笔记见：论文笔记&翻译——Enhanced LSTM for Natural Language Inference(ESIM)

ESIM主要分为三部分：input encoding，local inference modeling 和 inference composition。如上图所示，ESIM 是左边一部分, 如下图所示

三部分简要代码如下：

1. input encoding

1.1 原理

\bar{a}_i = BiLSTM(a, i), \forall i \in [1, 2, ..., l_a]

\bar{b}_j = BiLSTM(b, j), \forall j \in [1, 2, ..., l_b]

1.2 实现

i1 = Input(shape=(SentenceLen,), dtype='float32')
i2 = Input(shape=(SentenceLen,), dtype='float32')

x1 = Embedding([CONFIG])(i1)
x2 = Embedding([CONFIG])(i2)

x1 = Bidirectional(LSTM(300, return_sequences=True))(x1)
x2 = Bidirectional(LSTM(300, return_sequences=True))(x2)

2. local inference modeling

2.1 原理

\hat{a_i} = \sum_{j=1}^{l_b} \frac{\exp{e_{ij}}}{\sum_{k=1}^{l_b} \exp(e_{ik})} \bar{b}, \forall i \in [1, 2, ..., l_a]

\hat{b_j} = \sum_{i=1}^{l_a} \frac{\exp{e_{ij}}}{\sum_{k=1}^{l_a} \exp(e_{kj})} \bar{a}, \forall j \in [1, 2, ..., l_b]

2.2 实现

e = Dot(axes=2)([x1, x2])
e1 = Softmax(axis=2)(e)
e2 = Softmax(axis=1)(e)
e1 = Lambda(K.expand_dims, arguments={'axis' : 3})(e1)
e2 = Lambda(K.expand_dims, arguments={'axis' : 3})(e2)

_x1 = Lambda(K.expand_dims, arguments={'axis' : 1})(x2)
_x1 = Multiply()([e1, _x1])
_x1 = Lambda(K.sum, arguments={'axis' : 2})(_x1)
_x2 = Lambda(K.expand_dims, arguments={'axis' : 2})(x1)
_x2 = Multiply()([e2, _x2])
_x2 = Lambda(K.sum, arguments={'axis' : 1})(_x2)

3. inference composition

3.1 原理

m_a = [\bar{a}; \hat{a}; \bar{a} - \hat{a}; \bar{a} \odot \hat{a}]

m_b = [\bar{b}; \hat{b}; \bar{b} - \hat{b}; \bar{b} \odot \hat{b}]

v_{a,i} = BiLSTM(m_a, i)

v_{b,j} = BiLSTM(m_b, j)

v_{a,ave} = \sum_{i=1}^{l_a} \frac{v_{a,i}}{l_a}

v_{a,max} = \max_{i=1}^{l_a} v_{a,i}

v_{b,ave} = \sum_{j=1}^{l_b} \frac{v_{b,j}}{l_b}

v_{b,max} = \max_{j=1}^{l_b} v_{b,j}

v = [v_{a,ave}; v_{a,max}; v_{b,ave}; v_{b,max} ]

3.2 实现

m1 = Concatenate()([x1, _x1, Subtract()([x1, _x1]), Multiply()([x1, _x1])])
m2 = Concatenate()([x2, _x2, Subtract()([x2, _x2]), Multiply()([x2, _x2])])

y1 = Bidirectional(LSTM(300, return_sequences=True))(m1)
y2 = Bidirectional(LSTM(300, return_sequences=True))(m2)

mx1 = Lambda(K.max, arguments={'axis' : 1})(y1)
av1 = Lambda(K.mean, arguments={'axis' : 1})(y1)
mx2 = Lambda(K.max, arguments={'axis' : 1})(y2)
av2 = Lambda(K.mean, arguments={'axis' : 1})(y2)

y = Concatenate()([av1, mx1, av2, mx2])
y = Dense(1024, activation='tanh')(y)
y = Dropout(0.5)(y)
y = Dense(1024, activation='tanh')(y)
y = Dropout(0.5)(y)
y = Dense(2, activation='softmax')(y)

本文参与腾讯云自媒体同步曝光计划，分享自作者个人站点/博客。

原始发表：2018/12/13 ，如有侵权请联系 cloudcommunity@tencent.com 删除

机器学习