我不知道如何在Windows上使用变压器-cli。我让它在Google Colab上工作,同时也在使用它。
编辑
下面是我正在经历的过程,我所期待的,以及正在发生的事情:
--我在Windows上(括号是我在CMD中键入的确切命令)
安装transformers==2.8.0)I I transformers==2.8.0 (pip transformers==2.8.0)尝试运行变压器-cli,如Huggingface网站(Transformers)解释的那样
I get:
'transformers-cli' is not recognized as an interna
我正在尝试执行huggingface网站的示例代码:
from transformers import GPTJTokenizer, TFGPTJModel
import tensorflow as tf
tokenizer = GPTJTokenizer.from_pretrained("EleutherAI/gpt-j-6B")
model = TFGPTJModel.from_pretrained("EleutherAI/gpt-j-6B")
inputs = tokenizer("Hello, my dog is cute", ret
假设我成功地训练了10个时代的一些训练数据的模型。那么,我如何才能进入相同的模式,并继续训练10个时代呢?
建议“您需要通过超参数指定检查点输出路径”-->如何指定?
# define my estimator the standard way
huggingface_estimator = HuggingFace(
entry_point='train.py',
source_dir='./scripts',
instance_type='ml.p3.2xlarge',
instance_count=1,
我尝试从木星笔记本中的HuggingFace文档中执行标准的介绍示例:
from transformers import pipeline
classifier = pipeline("sentiment-analysis")
classifier("I've been waiting for a HuggingFace course my whole life.")
导入管道方法显然有效--没有错误消息。如果在下一行中声明分类器,则会得到以下错误:
/var/folders/m_/sn4z8b8s6676slgsrc3smg7w0000gn/T/ipy
我使用pytorch训练huggingface-transformers模型,但每个时期,总是输出警告:
The current process just got forked. Disabling parallelism to avoid deadlocks... To disable this warning, please explicitly set TOKENIZERS_PARALLELISM=(true | false)
如何禁用此警告?
我一直在尝试使用预训练模型。使用collab模板中默认的所有内容,使用从huggingface/pytorch-transformers到bert-base-uncased的torch.hub.load()作为“模型”
代码示例
import torch
model = torch.hub.load('huggingface/pytorch-transformers', 'model', 'bert-base-uncased') # Download model and configuration from S3 and cache.
我看
在转移到sagemaker进行实际培训之前,我正在尝试在本地开发sagemaker.huggingface.HuggingFace。我设了一个
HF_estimator = HuggingFace(entry_point='train.py', instance_type='local' ...)
并被称为HF_estimator.fit()
在train.py中,im简单地打印和退出以查看它是否能工作。然而,我遇到了这样的情况:
ValueError: Unsupported processor: cpu. You may need to upgrade yo
我想用huggingface做中文文本相似度: tokenizer = BertTokenizer.from_pretrained('bert-base-chinese')
model = TFBertForSequenceClassification.from_pretrained('bert-base-chinese') 它不工作,系统报告错误: Some weights of the model checkpoint at bert-base-chinese were not used when initializing TFBertForSequenc
我正在学习自然语言编程,遵循HuggingFace https://huggingface.co/transformers/custom_datasets.html#sequence-classification-with-imdb-reviews的序列分类教程,原始代码运行没有问题。但是,当我尝试加载一个不同的记号赋予器时,出现以下警告: Asking to truncate to max_length but no maximum length is provided and the model has no predefined maximum length. Default to n
给出了一个简单的神经网络,如:
import torch.nn as nn
net = nn.Sequential(
nn.Linear(3, 4),
nn.Sigmoid(),
nn.Linear(4, 1),
nn.Sigmoid()
).to(device)
如何将其转换为Huggingface 对象?
我们的目标是将Pytorch nn.Module对象从nn.Sequential转换为Huggingface PreTrainedModel对象,然后运行如下所示:
import torch.nn as nn
from tra
到今天为止,我一直用的是变压器。但是,当我今天导入包时,我收到了以下错误消息:
In Transformers v4.0.0, the default path to cache downloaded models changed from '~/.cache/torch/transformers' to '~/.cache/huggingface/transformers'. Since you don't seem to have overridden and '~/.cache/torch/transformers' is a di
我正在尝试使用huggingface的模型来实现一个QA系统。我不理解的一件事是,当我没有具体说明我正在使用哪个预先训练好的模型进行问答时,这个模型是随机选择的吗? from transformers import pipeline
# Allocate a pipeline for question-answering
question_answerer = pipeline('question-answering')
question_answerer({
'question': 'What is the name of the r
我正在尝试使用Bert预训练模型(bert-large-uncased with word-masking)我使用Huggingface来尝试它我第一次使用这段代码 m = TFBertLMHeadModel.from_pretrained("bert-large-cased-whole-word-masking")
logits = m(tokenizer("hello world [MASK] like it",return_tensors="tf")["input_ids"]).logits 在应用softmax之后,我使
model page提供了关于如何使用模型的以下代码片段: from transformers import M2M100ForConditionalGeneration, M2M100Tokenizer
hi_text = "जीवन एक चॉकलेट बॉक्स की तरह है।"
chinese_text = "生活就像一盒巧克力。"
model = M2M100ForConditionalGeneration.from_pretrained("facebook/m2m100_1.2B")
tokenizer = M2M100T