只要我检查了所有依赖项都已安装,并且在范围内(见下面),我还尝试卸载-安装可疑的有问题的库(烧瓶,请求)。到目前为止,我还没有使用venv,但如果这可能是一些libs的问题,特别是在进口过程中损坏的烧瓶),我也不知道这个Py版本是否会出现问题。
在我第一次尝试的Windows上,WSL2上的错误没有发生,手动导入烧瓶和HTTPAdapter运行,即from flask import Flask
和from requests.adapters import HTTPAdapter
返回很好。
在WSL中,>>> import requests
返回,但from requests.adapters import HTTPAdapter
与clip_retrieval错误一样是错误的。
(关于Win,最后的障碍可能是(据我所发现) BLAS和OpenMP缺少二进制文件/正确的安装/连接/构建( from . import _swigfaiss ...
)。我试着修复了一段时间,甚至手动复制了.dlls (libblas.dll,libiomp5md.dll,liblapack.dll),但是错误没有改变,于是我放弃了,尝试了WSL。
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import clip_retrieval
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python3.8/dist-packages/clip_retrieval/__init__.py", line 3, in <module>
from .clip_back import clip_back
File "/usr/local/lib/python3.8/dist-packages/clip_retrieval/clip_back.py", line 5, in <module>
from flask import Flask, request, make_response
ImportError: cannot import name 'Flask' from 'flask' (/home/tosh/.local/lib/python3.8/site-packages/flask/__init__.py)
Error in sys.excepthook:
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/apport_python_hook.py", line 72, in apport_excepthook
from apport.fileutils import likely_packaged, get_recent_crashes
File "/usr/lib/python3/dist-packages/apport/__init__.py", line 5, in <module>
from apport.report import Report
File "/usr/lib/python3/dist-packages/apport/report.py", line 32, in <module>
import apport.fileutils
File "/usr/lib/python3/dist-packages/apport/fileutils.py", line 12, in <module>
import os, glob, subprocess, os.path, time, pwd, sys, requests_unixsocket
File "/usr/lib/python3/dist-packages/requests_unixsocket/__init__.py", line 4, in <module>
from .adapters import UnixAdapter
File "/usr/lib/python3/dist-packages/requests_unixsocket/adapters.py", line 3, in <module>
from requests.adapters import HTTPAdapter
ImportError: cannot import name 'HTTPAdapter' from 'requests.adapters' (/home/tosh/.local/lib/python3.8/site-packages/requests/adapters.py)
Original exception was:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python3.8/dist-packages/clip_retrieval/__init__.py", line 3, in <module>
from .clip_back import clip_back
File "/usr/local/lib/python3.8/dist-packages/clip_retrieval/clip_back.py", line 5, in <module>
from flask import Flask, request, make_response
ImportError: cannot import name 'Flask' from 'flask' (/home/tosh/.local/lib/python3.8/site-packages/flask/__init__.py)
>>>
pip3 list
Package Version
------------------------ --------------------
aiohttp 3.8.3
aiosignal 1.2.0
albumentations 1.3.0
aniso8601 9.0.1
async-timeout 4.0.2
attrs 19.3.0
autofaiss 2.15.3
Automat 0.8.0
blinker 1.4
braceexpand 0.1.7
certifi 2019.11.28
chardet 3.0.4
charset-normalizer 2.1.1
click 8.1.3
clip-anytorch 2.5.0
clip-retrieval 2.35.1
cloud-init 22.3.4
colorama 0.4.3
command-not-found 0.3
configobj 5.0.6
constantly 15.1.0
cryptography 2.8
cupshelpers 1.0
cycler 0.11.0
dataclasses 0.6
dbus-python 1.2.16
defer 1.0.6
Distance 0.1.3
distro 1.4.0
distro-info 0.23ubuntu1
docker-pycreds 0.4.0
embedding-reader 1.5.0
entrypoints 0.3
ExifRead-nocycle 3.0.1
faiss-cpu 1.7.2
filelock 3.8.0
fire 0.4.0
Flask 2.2.2
Flask-Cors 3.0.10
Flask-RESTful 0.3.9
fonttools 4.28.5
frozenlist 1.3.1
fsspec 2022.1.0
ftfy 6.1.1
gitdb 4.0.9
GitPython 3.1.29
h5py 3.7.0
httplib2 0.14.0
huggingface-hub 0.10.1
hyperlink 19.0.0
idna 2.8
imageio 2.22.4
img2dataset 1.33.0
importlib-metadata 5.0.0
incremental 16.10.1
itsdangerous 2.1.2
Jinja2 3.1.2
joblib 1.2.0
jsonpatch 1.22
jsonpointer 2.0
jsonschema 3.2.0
keyring 18.0.1
kiwisolver 1.3.2
language-selector 0.1
launchpadlib 1.10.13
lazr.restfulclient 0.14.2
lazr.uri 1.0.3
macaroonbakery 1.3.1
MarkupSafe 2.1.1
matplotlib 3.5.1
more-itertools 4.2.0
multidict 6.0.2
multilingual-clip 1.0.10
netifaces 0.10.4
networkx 2.8.8
nltk 3.7
numpy 1.22.0
nvidia-cublas-cu11 11.10.3.66
nvidia-cuda-nvrtc-cu11 11.7.99
nvidia-cuda-runtime-cu11 11.7.99
nvidia-cudnn-cu11 8.5.0.96
oauthlib 3.1.0
onnx 1.11.0
open-clip-torch 2.3.1
opencv-python 4.6.0.66
opencv-python-headless 4.6.0.66
packaging 21.3
pandas 1.5.1
pathtools 0.1.2
pexpect 4.6.0
Pillow 9.0.0
pip 20.0.2
prometheus-client 0.15.0
promise 2.3
protobuf 3.19.4
psutil 5.5.1
pyarrow 7.0.0
pyasn1 0.4.2
pyasn1-modules 0.2.1
pybind11 2.9.1
pycairo 1.16.2
pycups 1.9.73
PyGObject 3.36.0
PyHamcrest 1.9.0
PyJWT 1.7.1
pymacaroons 0.13.0
PyNaCl 1.3.0
pyOpenSSL 19.0.0
pyparsing 3.0.6
pyRFC3339 1.1
pyrsistent 0.15.5
pyserial 3.4
python-apt 2.0.0+ubuntu0.20.4.8
python-dateutil 2.8.2
python-debian 0.1.36ubuntu1
pytz 2022.6
PyWavelets 1.4.1
PyYAML 5.3.1
qudida 0.0.4
regex 2022.10.31
requests 2.28.1
requests-unixsocket 0.2.0
scikit-image 0.19.3
scikit-learn 1.1.3
scipy 1.9.3
SecretStorage 2.3.1
sentence-transformers 2.2.2
sentencepiece 0.1.97
sentry-sdk 1.10.1
service-identity 18.1.0
setproctitle 1.3.2
setuptools 60.9.3
shortuuid 1.0.9
simplejson 3.16.0
six 1.14.0
smmap 5.0.0
sos 4.4
ssh-import-id 5.10
systemd-python 234
termcolor 2.1.0
threadpoolctl 3.1.0
tifffile 2022.10.10
tokenizers 0.13.2
torch 1.13.0
torchvision 0.14.0
tqdm 4.64.1
transformers 4.24.0
Twisted 18.9.0
typing-extensions 4.1.1
ubuntu-advantage-tools 27.11.3
ufw 0.36
unattended-upgrades 0.1
urllib3 1.26.11
wadllib 1.3.3
wandb 0.12.21
wcwidth 0.2.5
webdataset 0.1.103
Werkzeug 2.2.2
wheel 0.34.2
yarl 1.8.1
zipp 1.0.0
zope.interface 4.7.1
https://github.com/rom1504/clip-retrieval/blob/main/requirements.txt
img2dataset>=1.25.5,<2
clip-anytorch>=2.5.0,<3
tqdm>=4.62.3,<5
fire>=0.4.0,<0.5.0
torch>=1.7.1,<2
torchvision>=0.10.1,<2
numpy>=1.19.5,<2
faiss-cpu>=1.7.2,<2
flask>=2.0.3,<3
flask_restful>=0.3.9,<1
flask_cors>=3.0.10,<4
pandas>=1.1.5,<2
pyarrow>=6.0.1,<8
autofaiss>=2.9.6,<3
webdataset>=0.1.103,<0.2
h5py>=3.1.0,<4
prometheus-client>=0.13.1,<1
fsspec==2022.1.0
sentence-transformers>=2.2.0,<3
wandb>=0.12.10,<0.13
open-clip-torch>=2.0.0,<3.0.0
requests>=2.27.1,<3
aiohttp>=3.8.1,<4
multilingual-clip>=1.0.10,<2
编辑:这是通过安装Anaconda在Windows上修复的(感谢rom1504的建议,它应该使用venv -我试图避免使用一个)。然而,另一个错误有待解决。见:https://github.com/rom1504/clip-retrieval/issues/204
发布于 2022-11-09 23:34:56
我设法修复它,并在本地运行整个剪辑检索记事本示例,并从因特网访问本地隧道Node.js服务器。这是必须从WSL或Windows运行的代码:
导入现在完成。
>>> import clip_retrieval
>>> dir(clip_retrieval)
['__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__path__', '__spec__', 'clip_back', 'clip_end2end', 'clip_filter', 'clip_front', 'clip_index', 'clip_inference', 'ivf_metadata_ordering', 'load_clip']
解决方案:我迁移到Python3.10(可能不是严格需要的,IDK)并升级了pip。然后,在试图运行剪辑检索命令时,在导入_pustil时出现了另一个错误,即“部分导入”,这恰好是安装过程中的一个冲突,可能是将sudo apt-get install
和pip3 install
、python3.10 -m pip install
混合在一起,因为错误跟踪包括对两个不同目录的引用。
File "/usr/lib/python3/dist-packages/psutil/__init__.py", line 95, in <module>
from . import _pslinux as _psplatform
File "/usr/lib/python3/dist-packages/psutil/_pslinux.py", line 26, in <module>
from . import _psutil_linux as cext
ImportError: cannot import name '_psutil_linux' from partially initialized module 'psutil' (most likely due to a circular import) (/usr/lib/python3/dist-packages/psutil/__init__.py)
删除和重新安装冲突库存在问题,但使用sudo rm -r手动删除.对带有模块的文件夹进行psutil,然后用pip重新安装它,最后解决它。(有两个,这增加了混乱,一个是带有.python3 sudo rm -r /usr/lib/python3 3/dist-packages/psutil-5.5.1 sudo rm -r -r)。
img2dataset --url_list=myimglist.txt --output_folder=image_folder --thread_count=4 --image_
size=256
Starting the downloading of this file
Sharding file number 1 of 1 called /mnt/z/myimglist.txt
0it [00:00, ?it/s]File sharded in 0 shards
Downloading starting now, check your bandwidth speed (with bwm-ng)your cpu (with htop), and your disk usage (with iotop)!
0it [00:00, ?it/s]
myimglist.txt
https://placekitten.com/200/305
https://placekitten.com/200/304
https://placekitten.com/200/303
https://placekitten.com/200/302
https://placekitten.com/150/235
https://placekitten.com/120/120
调用剪辑检索时,它将下载
clip-retrieval inference --input_dataset image_folder --output_folder embedding_folder
The number of samples has been estimated to be 6
Starting the worker
dataset is 12
Starting work on task 0
100%|███████████████████████████████████████| 354M/354M [01:35<00:00, 3.72MiB/s]
/home/xxxx/.local/lib/python3.10/site-packages/clip/clip.py:160: FutureWarning: 'torch.onnx._patch_torch._node_getitem' is deprecated in version 1.13 and will be removed in version 1.14. Please Internally use '_node_get' in symbolic_helper instead..
if "value" in node.attributeNames() and str(node["value"]).startswith("cuda"):
/home/xxxx/.local/lib/python3.10/site-packages/clip/clip.py:186: FutureWarning: 'torch.onnx._patch_torch._node_getitem' is deprecated in version 1.13 and will be removed in version 1.14. Please Internally use '_node_get' in symbolic_helper instead..
if inputs[i].node()["value"] == 5:
warming up with batch size 256 on cpu
done warming up in 56.720762968063354s
/home/tosh/.local/lib/python3.10/site-packages/torch/utils/data/dataloader.py:554: UserWarning: This DataLoader will create 8 worker processes in total. Our suggested max number of worker in current system is 4, which is smaller than what this DataLoader is going to create. Please be aware that excessive worker creation might get DataLoader running slow or even freeze, lower the worker number to avoid potential slowness/freeze if necessary.
warnings.warn(_create_warning_msg(
sample_per_sec 1 ; sample_count 6
..。
Windows
如果这个信息对任何人都有用的话,我意识到我试图在CLI中导入clip_retrieval (在WSL上我最初也是这样做的),而示例笔记本中的示例用法是将它作为命令调用。img2dataset成功地:
<frozen importlib._bootstrap>:228: RuntimeWarning: scipy._lib.messagestream.MessageStream size changed, may indicate binary incompatibility. Expected 56 from C header, got 64 from PyObject
Starting the downloading of this file
Sharding file number 1 of 1 called z:/list.txt
0it [00:00, ?it/s]File sharded in 1 shards
Downloading starting now, check your bandwidth speed (with bwm-ng)your cpu (with htop), and your disk usage (with iotop)!
<frozen importlib._bootstrap>:228: RuntimeWarning: scipy._lib.messagestream.MessageStream size changed, may indicate binary incompatibility. Expected 56 from C header, got 64 from PyObject
<frozen importlib._bootstrap>:228: RuntimeWarning: scipy._lib.messagestream.MessageStream size changed, may indicate binary incompatibility. Expected 56 from C header, got 64 from PyObject
1it [00:07, 7.66s/it]
worker - success: 1.000 - failed to download: 0.000 - failed to resize: 0.000 - images per sec: 8 - count: 18
total - success: 1.000 - failed to download: 0.000 - failed to resize: 0.000 - images per sec: 8 - count: 18
正确调用剪辑-检索现在返回了另一个错误: JAX要求,而且它似乎必须从源代码构建。所以可能只是JAX,而不是OpenMP和BLAS,IDK。
clip-retrieval
<frozen importlib._bootstrap>:228: RuntimeWarning: scipy._lib.messagestream.MessageStream size changed, may indicate binary incompatibility. Expected 56 from C header, got 64 from PyObject
Traceback (most recent call last):
File "C:\Users\toshb\AppData\Local\Programs\Python\Python39\lib\site-packages\jax\_src\lib\__init__.py", line 38, in <module>
import jaxlib as jaxlib
ModuleNotFoundError: No module named 'jaxlib'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "C:\Users\toshb\AppData\Local\Programs\Python\Python39\lib\runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "C:\Users\toshb\AppData\Local\Programs\Python\Python39\lib\runpy.py", line 87, in _run_code
exec(code, run_globals)
File "C:\Users\toshb\AppData\Local\Programs\Python\Python39\Scripts\clip-retrieval.exe\__main__.py", line 4, in <module>
File "C:\Users\toshb\AppData\Roaming\Python\Python39\site-packages\clip_retrieval\__init__.py", line 6, in <module>
from .clip_inference.main import main as clip_inference
File "C:\Users\toshb\AppData\Roaming\Python\Python39\site-packages\clip_retrieval\clip_inference\__init__.py", line 3, in <module>
from .main import main as clip_inference
File "C:\Users\toshb\AppData\Roaming\Python\Python39\site-packages\clip_retrieval\clip_inference\main.py", line 10, in <module>
from clip_retrieval.clip_inference.distributor import PysparkDistributor, SequentialDistributor
File "C:\Users\toshb\AppData\Roaming\Python\Python39\site-packages\clip_retrieval\clip_inference\distributor.py", line 5, in <module>
from .worker import worker
File "C:\Users\toshb\AppData\Roaming\Python\Python39\site-packages\clip_retrieval\clip_inference\worker.py", line 15, in <module>
from clip_retrieval.clip_inference.mapper import ClipMapper
File "C:\Users\toshb\AppData\Roaming\Python\Python39\site-packages\clip_retrieval\clip_inference\mapper.py", line 5, in <module>
from sentence_transformers import SentenceTransformer
File "C:\Users\toshb\AppData\Roaming\Python\Python39\site-packages\sentence_transformers\__init__.py", line 3, in <module>
from .datasets import SentencesDataset, ParallelSentencesDataset
File "C:\Users\toshb\AppData\Roaming\Python\Python39\site-packages\sentence_transformers\datasets\__init__.py", line 3, in <module>
from .ParallelSentencesDataset import ParallelSentencesDataset
File "C:\Users\toshb\AppData\Roaming\Python\Python39\site-packages\sentence_transformers\datasets\ParallelSentencesDataset.py", line 4, in <module>
from .. import SentenceTransformer
File "C:\Users\toshb\AppData\Roaming\Python\Python39\site-packages\sentence_transformers\SentenceTransformer.py", line 11, in <module>
import transformers
File "C:\Users\toshb\AppData\Roaming\Python\Python39\site-packages\transformers\__init__.py", line 30, in <module>
from . import dependency_versions_check
File "C:\Users\toshb\AppData\Roaming\Python\Python39\site-packages\transformers\dependency_versions_check.py", line 17, in <module>
from .utils.versions import require_version, require_version_core
File "C:\Users\toshb\AppData\Roaming\Python\Python39\site-packages\transformers\utils\__init__.py", line 34, in <module>
from .generic import (
File "C:\Users\toshb\AppData\Roaming\Python\Python39\site-packages\transformers\utils\generic.py", line 36, in <module>
import jax.numpy as jnp
File "C:\Users\toshb\AppData\Local\Programs\Python\Python39\lib\site-packages\jax\__init__.py", line 35, in <module>
from jax import config as _config_module
File "C:\Users\toshb\AppData\Local\Programs\Python\Python39\lib\site-packages\jax\config.py", line 17, in <module>
from jax._src.config import config
File "C:\Users\toshb\AppData\Local\Programs\Python\Python39\lib\site-packages\jax\_src\config.py", line 29, in <module>
from jax._src import lib
File "C:\Users\toshb\AppData\Local\Programs\Python\Python39\lib\site-packages\jax\_src\lib\__init__.py", line 40, in <module>
raise ModuleNotFoundError(
ModuleNotFoundError: jax requires jaxlib to be installed. See https://github.com/google/jax#installation for installation instructions.
https://stackoverflow.com/questions/74367611
复制相似问题