如何解决拥抱面错误ImportError:无法从“Transferers.utils”导入名称“is_tokenizers_available”？
如何解决拥抱面错误ImportError:无法从“Transferers.utils”导入名称“is_tokenizers_available”？
提问于 2022-10-03 19:24:36
我试着用ViTT转染。我得到了下面的代码错误：
from pathlib import Path
import torchvision
from typing import Callable
root = Path("~/data/").expanduser()
# root = Path(".").expanduser()
train = torchvision.datasets.CIFAR100(root=root, train=True, download=True)
test = torchvision.datasets.CIFAR100(root=root, train=False, download=True)
img2tensor: Callable = torchvision.transforms.ToTensor()
from transformers import ViTFeatureExtractor
feature_extractor = ViTFeatureExtractor.from_pretrained("google/vit-base-patch16-224-in21k")
x, y = train_ds[0]
print(f'{y=}')
print(f'{type(x)=}')
x = img2tensor(x)
x = x.unsqueeze(0) # add batch size 1
out_cls: ImageClassifierOutput = model(x)
print(f'{out_cls.logits=}')错误
Files already downloaded and verified
Files already downloaded and verified
Traceback (most recent call last):
File "/Users/brandomiranda/opt/anaconda3/envs/meta_learning/lib/python3.9/code.py", line 90, in runcode
exec(code, self.locals)
File "<input>", line 11, in <module>
File "/Applications/PyCharm.app/Contents/plugins/python/helpers/pydev/_pydev_bundle/pydev_import_hook.py", line 21, in do_import
module = self._system_import(name, *args, **kwargs)
File "/Users/brandomiranda/opt/anaconda3/envs/meta_learning/lib/python3.9/site-packages/transformers/__init__.py", line 30, in <module>
from . import dependency_versions_check
File "/Applications/PyCharm.app/Contents/plugins/python/helpers/pydev/_pydev_bundle/pydev_import_hook.py", line 21, in do_import
module = self._system_import(name, *args, **kwargs)
File "/Users/brandomiranda/opt/anaconda3/envs/meta_learning/lib/python3.9/site-packages/transformers/dependency_versions_check.py", line 36, in <module>
from .utils import is_tokenizers_available
ImportError: cannot import name 'is_tokenizers_available' from 'transformers.utils' (/Users/brandomiranda/opt/anaconda3/envs/meta_learning/lib/python3.9/site-packages/transformers/utils/__init__.py)我试着升级所有东西，但还是失败了。升级命令：
/Users/brandomiranda/opt/anaconda3/envs/meta_learning/bin/python -m
pip install --upgrade pip
pip install --upgrade pip
pip install --upgrade huggingface-hub
pip install --upgrade transformers
pip install --upgrade huggingface-hub
pip install --upgrade datasets
pip install --upgrade tokenizers
pip install pytorch-transformers
pip install --upgrade torch
pip install --upgrade torchvision
pip install --upgrade torchtext
pip install --upgrade torchaudio
# pip install --upgrade torchmeta
pip uninstall torchmeta为什么以及如何修复它？
和平执行方案清单：
(meta_learning) ❯ pip list
Package Version Editable project location
------------------------------------------------- ---------- ------------------------------------------------------------------------------
absl-py 1.0.0
aiohttp 3.8.1
aiosignal 1.2.0
antlr4-python3-runtime 4.8
argcomplete 2.0.0
async-timeout 4.0.1
attrs 21.4.0
automl-meta-learning 0.1.0 /Users/brandomiranda/automl-meta-learning/automl-proj-src
bcj-cffi 0.5.1
boto 2.49.0
boto3 1.24.85
botocore 1.27.85
Bottleneck 1.3.4
Brotli 1.0.9
brotlicffi 1.0.9.2
brotlipy 0.7.0
cachetools 4.2.4
certifi 2022.9.14
cffi 1.15.1
charset-normalizer 2.0.9
cherry-rl 0.1.4
click 8.0.3
cloudpickle 2.0.0
colorama 0.4.4
configparser 5.2.0
conllu 4.4.1
crcmod 1.7
cryptography 37.0.1
cycler 0.11.0
Cython 0.29.25
dataclasses 0.6
datasets 2.5.1
dill 0.3.4
diversity-for-predictive-success-of-meta-learning 0.0.1 /Users/brandomiranda/diversity-for-predictive-success-of-meta-learning/div_src
docker-pycreds 0.4.0
editdistance 0.6.0
et-xmlfile 1.1.0
fairseq 0.10.0
fastcluster 1.2.4
fasteners 0.17.3
filelock 3.6.0
fonttools 4.28.3
frozenlist 1.2.0
fsspec 2022.7.1
gcs-oauth2-boto-plugin 3.0
gitdb 4.0.9
GitPython 3.1.24
google-apitools 0.5.32
google-auth 2.3.3
google-auth-oauthlib 0.4.6
google-reauth 0.1.1
grpcio 1.42.0
gsutil 5.6
gym 0.21.0
h5py 3.6.0
higher 0.2.1
httplib2 0.20.4
huggingface-hub 0.10.0
hydra-core 1.1.1
idna 3.3
importlib-metadata 4.11.3
jmespath 1.0.1
joblib 1.1.0
kiwisolver 1.3.2
lark-parser 0.12.0
learn2learn 0.1.7
lxml 4.8.0
Markdown 3.3.6
matplotlib 3.5.1
mkl-fft 1.3.1
mkl-random 1.2.2
mkl-service 2.4.0
monotonic 1.6
multidict 5.2.0
multiprocess 0.70.12.2
multivolumefile 0.2.3
munkres 1.1.4
networkx 2.6.3
numexpr 2.8.1
numpy 1.21.5
oauth2client 4.1.3
oauthlib 3.1.1
omegaconf 2.1.1
openpyxl 3.0.10
ordered-set 4.0.2
packaging 21.3
pandas 1.4.2
pathtools 0.1.2
Pillow 9.0.1
pip 22.2.2
plotly 5.4.0
portalocker 2.3.2
progressbar2 3.55.0
promise 2.3
protobuf 3.19.1
psutil 5.8.0
py7zr 0.16.1
pyarrow 9.0.0
pyasn1 0.4.8
pyasn1-modules 0.2.8
pycparser 2.21
pycryptodomex 3.15.0
pyOpenSSL 22.0.0
pyparsing 3.0.6
pyppmd 0.16.1
PySocks 1.7.1
python-dateutil 2.8.2
python-utils 2.5.6
pytorch-transformers 1.2.0
pytz 2021.3
pyu2f 0.1.5
PyYAML 6.0
pyzstd 0.14.4
qpth 0.0.15
regex 2021.11.10
requests 2.28.1
requests-oauthlib 1.3.0
responses 0.18.0
retry-decorator 1.1.1
rsa 4.7.2
s3transfer 0.6.0
sacrebleu 2.0.0
sacremoses 0.0.46
scikit-learn 1.0.1
scipy 1.7.3
seaborn 0.11.2
sentencepiece 0.1.97
sentry-sdk 1.5.1
setproctitle 1.2.2
setuptools 58.0.4
shortuuid 1.0.8
six 1.16.0
sklearn 0.0
smmap 5.0.0
subprocess32 3.5.4
tabulate 0.8.9
tenacity 8.0.1
tensorboard 2.7.0
tensorboard-data-server 0.6.1
tensorboard-plugin-wit 1.8.0
termcolor 1.1.0
texttable 1.6.4
threadpoolctl 3.0.0
tokenizers 0.13.0
torch 1.12.1
torchaudio 0.12.1
torchtext 0.13.1
torchvision 0.13.1
tornado 6.1
tqdm 4.62.3
transformers 4.22.2
typing_extensions 4.3.0
ultimate-anatome 0.1.1 /Users/brandomiranda/ultimate-anatome
ultimate-aws-cv-task2vec 0.0.1 /Users/brandomiranda/ultimate-aws-cv-task2vec
ultimate-utils 0.6.1 /Users/brandomiranda/ultimate-utils/ultimate-utils-proj-src
urllib3 1.26.11
wandb 0.13.3
Werkzeug 2.0.2
wheel 0.37.0
xxhash 2.0.2
yarl 1.8.1
yaspin 2.1.0
zipp 3.8.0拥抱面部相关组织：
- https://github.com/huggingface/transformers/issues/15062
- https://github.com/huggingface/tokenizers/issues/120
- https://github.com/huggingface/tokenizers/issues/585
- https://github.com/huggingface/transformers/issues/11262
- 所以相关：调度器‘“
- 高频讨论：https://discuss.huggingface.co/t/how-to-resolve-the-hugging-face-error-importerror-cannot-import-name-is-tokenizers-available-from-transformers-utils/23957
- reddit：重大恐怖/
- https://github.com/huggingface/tokenizers/issues/1080
回答 2
Stack Overflow用户
发布于 2022-10-11 06:04:35
你没有在你的问题中给出模型。
使用google，我可以轻松地运行模型openai/剪辑-vit-底部-补丁32
from PIL import Image
import requests
from transformers import CLIPProcessor, CLIPModel
model = CLIPModel.from_pretrained("openai/clip-vit-base-patch32")
processor = CLIPProcessor.from_pretrained("openai/clip-vit-base-patch32")
url = "http://images.cocodataset.org/val2017/000000039769.jpg"
image = Image.open(requests.get(url, stream=True).raw)
inputs = processor(text=["a photo of a cat", "a photo of a dog"],
images=image,
return_tensors="pt",
padding=True)
outputs = model(**inputs)
logits_per_image = outputs.logits_per_image # this is the image-text similarity score
probs = logits_per_image.softmax(dim=1) # we can take the softmax to get the label probabilities还有google/vit-base-patch16 16-224-vit 21k
from transformers import ViTFeatureExtractor, ViTModel
import torch
feature_extractor = ViTFeatureExtractor.from_pretrained("google/vit-base-patch16-224-in21k")
model = ViTModel.from_pretrained("google/vit-base-patch16-224-in21k")
inputs = feature_extractor(image, return_tensors="pt")
with torch.no_grad():
outputs = model(**inputs)
last_hidden_states = outputs.last_hidden_state
list(last_hidden_states.shape)如果您想在自己的任务中使用它，那么最后一个示例是一个很好的起点，因为您可以将last_hidden_state输出传递给您将要培训的自定义模型。
然而，如果您直接尝试这些模型，cifar-100数据集，您将遇到形状错配问题。模型需要244x244张图像，但cifar数据集是32x32张图像。
Stack Overflow用户
发布于 2022-10-12 03:10:10
如果定义了这个模型，我会说它看起来像一个anaconda问题，我建议使用一个不同的虚拟环境来测试这个问题，例如pipenv。另一种选择是使用timm库，它也有用于图像分类的模型。
页面原文内容由Stack Overflow提供。腾讯云小微IT领域专用引擎提供翻译支持
原文链接：
https://stackoverflow.com/questions/73939929
复制相关文章
相似问题
腾讯云开发者
