等效于Transformers 2.5.0中的记号器()？
等效于Transformers 2.5.0中的记号器()？
提问于 2022-07-26 16:55:11
我正在尝试将以下代码转换为使用Transformer2.5.0。正如所写的，它在4.18.0版本中工作，但不使用2.5.0。
# Converting pretrained BERT classification model to regression model
# i.e. extracting base model and swapping out heads
from transformers import BertTokenizer, BertModel, BertConfig, BertForMaskedLM, BertForSequenceClassification, AutoConfig, AutoModelForTokenClassification
import torch
import numpy as np
old_model = BertForSequenceClassification.from_pretrained("textattack/bert-base-uncased-yelp-polarity")
model = BertForSequenceClassification.from_pretrained("bert-base-uncased", num_labels=1)
model.bert = old_model.bert
# Ensure that model parameters are equivalent except for classifier head layer
for param_name in model.state_dict():
if 'classifier' not in param_name:
sub_param, full_param = model.state_dict()[param_name], old_model.state_dict()[param_name] # type: torch.Tensor, torch.Tensor
assert (sub_param.cpu().numpy() == full_param.cpu().numpy()).all(), param_name
tokenizer = BertTokenizer.from_pretrained("textattack/bert-base-uncased-yelp-polarity")
inputs = tokenizer("Hello, my dog is cute", return_tensors="pt")
with torch.no_grad():
logits = model(**inputs).logits
output_value = np.array(logits)[0][0]
print(output_value)不能使用变压器2.5.0调用令牌程序，因此如下所示：
TypeError Traceback (most recent call last)
<ipython-input-1-d83f0d613f4b> in <module>
19
20
---> 21 inputs = tokenizer("Hello, my dog is cute", return_tensors="pt")
22
23 with torch.no_grad():
TypeError: 'BertTokenizer' object is not callable但是，试图用tokenizer.tokenize()替换令牌程序()，结果如下：
TypeError Traceback (most recent call last)
<ipython-input-2-1d431131eb87> in <module>
21
22 with torch.no_grad():
---> 23 logits = model(**inputs).logits
24
25 output_value = np.array(logits)[0][0]
TypeError: BertForSequenceClassification object argument after ** must be a mapping, not list任何帮助都将不胜感激。
解决方案
使用@cronoik建议的tokenizer.encode_plus()：
tokenized = tokenizer.encode_plus("Hello, my dog is cute", return_tensors="pt")
with torch.no_grad():
logits = model(**tokenized)
output_value = np.array(logits)[0]
print(output_value)Stack Overflow用户
回答已采纳
发布于 2022-07-26 17:57:04
遗憾的是，旧版本的文档已经损坏，但是您可以使用encode_plus，如下所示( encode_plus最古老的可用文档来自2.10.0)：
import torch
from transformers import BertTokenizer
t = BertTokenizer.from_pretrained("textattack/bert-base-uncased-yelp-polarity")
tokenized = t.encode_plus("Hello, my dog is cute", return_tensors='pt')
print(tokenized)输出：
{'input_ids': tensor([[ 101, 7592, 1010, 2026, 3899, 2003, 10140, 102]]),
'token_type_ids': tensor([[0, 0, 0, 0, 0, 0, 0, 0]]),
'attention_mask': tensor([[1, 1, 1, 1, 1, 1, 1, 1]])}页面原文内容由Stack Overflow提供。腾讯云小微IT领域专用引擎提供翻译支持
原文链接：
https://stackoverflow.com/questions/73127139
复制相关文章
相似问题
腾讯云开发者
