[Kaggle] Sentiment Analysis on Movie Reviews（BERT）

Michael阿明

发布于 2021-02-19 14:53:01

7510

发布于 2021-02-19 14:53:01

文章被收录于专栏：Michael阿明学习之路

文章目录

练习地址：https://www.kaggle.com/c/sentiment-analysis-on-movie-reviews

1. 预训练模型下载

下载地址 https://huggingface.co/bert-base-uncased/tree/main

模型下载很慢的话，我传到 csdn了，可以免费下载

存放在目录如./bert-base-uncased下

2. 数据集

数据集切分

train_csv = pd.read_csv("./train.tsv", sep='\t')
test_csv = pd.read_csv("./test.tsv", sep='\t')
train_csv.head(50)
# %%
test_csv.head()
# %%

# 切分出一些验证集，分层抽样
from sklearn.model_selection import StratifiedShuffleSplit

splt = StratifiedShuffleSplit(n_splits=1, test_size=0.2, random_state=1)
for train_idx, valid_idx in splt.split(train_csv, train_csv['Sentiment']):
    train_part = train_csv.loc[train_idx]
    valid_part = train_csv.loc[valid_idx]

y_train = train_part['Sentiment']
y_valid = valid_part['Sentiment']
X_train = train_part['Phrase']
X_valid = valid_part['Phrase']

X_test = test_csv['Phrase']
y_test = [0] * len(X_test)  # 测试集没有标签，这么处理方便代码处理
y_test = torch.LongTensor(y_test)  # 转成tensor

3. 加载预训练模型

from transformers import AutoTokenizer, BertForSequenceClassification

tokenizer = AutoTokenizer.from_pretrained("./bert-base-uncased")

# num_classes = 5 ， 5种情绪
pretrain_model = BertForSequenceClassification.from_pretrained("./bert-base-uncased", num_labels=num_classes)

编写自定义模型

class myModel(nn.Module):
    def __init__(self):
        super(myModel, self).__init__()
        self.pretrain_model = pretrain_model
        for param in self.pretrain_model.parameters():
            param.requires_grad = True

    def forward(self, x):
        context = x[0]
        mask = x[2]
        out = self.pretrain_model(context, attention_mask=mask)
        # out 的 size [batch_size, num_classes]
        out = torch.softmax(out.logits, 1) # 归一化 维度 1 为概率
        return out

注：其余数据处理、训练等代码跟前一篇完全一样

本文完整代码