Pip安装-内存耗尽-如何绕过它？
Pip安装-内存耗尽-如何绕过它？
提问于 2019-07-16 13:41:09
我正在尝试安装spaCy，一个用于自然语言处理的库。
root@vps:~# python3 -m pip install spacy
Collecting spacy
Using cached https://files.pythonhosted.org/packages/8f/ac/f7a56f8de3bd29384247babb327b7f3c0e3ddef40d1eff8ae993e6d4d2ba/spacy-2.1.6-cp35-cp35m-manylinux1_x86_64.whl
Installing collected packages: spacy
Killed过了一段时间，它就死了。我已经在堆栈溢出上寻找了答案，并尝试了，这是由于内存不足( VPS上可用1，2 Gb，无法扩展)。一个快速的tail -f /var/log/kern.log给了我Out of memory: Kill process 31729 (python3) score 558 or sacrifice child。
我已经尝试过添加--no-cache-dir，但是没有帮助。在手动复制文件时，似乎缺少了一些东西。有没有办法在低内存服务器上安装这个软件包？
编辑：python3 -m pip install spacy -vvv的完整日志
Converted retries value: Retry(total=5, connect=None, read=None, redirect=None) -> Retry(total=Retry(total=5, connect=None, read=None, redirect=None), connect=N one, read=None, redirect=None)
Converted retries value: Retry(total=5, connect=None, read=None, redirect=None) -> Retry(total=Retry(total=5, connect=None, read=None, redirect=None), connect=N one, read=None, redirect=None)
Collecting spacy
1 location(s) to search for versions of spacy:
* https://pypi.python.org/simple/spacy/
Getting page https://pypi.python.org/simple/spacy/
Looking up "https://pypi.python.org/simple/spacy/" in the cache
Returning cached "301 Moved Permanently" response (ignoring date and etag info rmation)
Looking up "https://pypi.org/simple/spacy/" in the cache
Current age based on date: 29458
Freshness lifetime from max-age: 600
Freshness lifetime from request max-age: 600
Starting new HTTPS connection (1): pypi.org
https://pypi.org:443 "GET /simple/spacy/ HTTP/1.1" 304 0
Analyzing links from page https://pypi.org/simple/spacy/
<!-- Found link, skipped linked (hundreds of lines) -->
Found link https://files.pythonhosted.org/packages/a7/84/f35f44560f0ab9ef23c 4a334a513f2b6c8a95d7b3347ef68a0247da63c08/spacy-2.1.6.tar.gz#sha256=b708bb582597 6220de78ad7e20e827912ca95653738a812cc2e5d194e3060ae8 (from https://pypi.org/simp le/spacy/) (requires-python:>=2.7,!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*), version: 2.1 .6
Using version 2.1.6 (newest of versions: 0.31, 0.32, 0.33, 0.40, 0.51, 0.52, 0 .60, 0.61, 0.62, 0.63, 0.64, 0.65, 0.67, 0.68, 0.70, 0.80, 0.81, 0.82, 0.83, 0.8 4, 0.85, 0.86, 0.87, 0.88, 0.89, 0.90, 0.91, 0.92, 0.93, 0.94, 0.95, 0.97, 0.98, 0.99, 0.100.0, 0.100.1, 0.100.2, 0.100.3, 0.100.4, 0.100.5, 0.100.6, 0.100.7, 0 .101.0, 1.0.1, 1.0.2, 1.0.3, 1.0.4, 1.0.5, 1.1.0, 1.1.1, 1.1.2, 1.2.0, 1.3.0, 1. 4.0, 1.5.0, 1.5.1, 1.6.0, 1.7.0, 1.7.1, 1.7.2, 1.7.3, 1.7.5, 1.8.0, 1.8.1, 1.8.2 , 1.9.0, 1.10.0, 1.10.1, 2.0.0, 2.0.1, 2.0.2, 2.0.3, 2.0.4, 2.0.5, 2.0.6, 2.0.7, 2.0.8, 2.0.9, 2.0.10, 2.0.11, 2.0.12, 2.0.13, 2.0.15, 2.0.16, 2.0.17, 2.0.18, 2 .1.0, 2.1.1, 2.1.2, 2.1.3, 2.1.4, 2.1.5, 2.1.6)
Looking up "https://files.pythonhosted.org/packages/8f/ac/f7a56f8de3bd29384247 babb327b7f3c0e3ddef40d1eff8ae993e6d4d2ba/spacy-2.1.6-cp35-cp35m-manylinux1_x86_6 4.whl" in the cache
Current age based on date: 98439
Freshness lifetime from max-age: 365000000
The response is "fresh", returning cached response
365000000 > 98439
Using cached https://files.pythonhosted.org/packages/8f/ac/f7a56f8de3bd2938424 7babb327b7f3c0e3ddef40d1eff8ae993e6d4d2ba/spacy-2.1.6-cp35-cp35m-manylinux1_x86_ 64.whl
Downloading from URL https://files.pythonhosted.org/packages/8f/ac/f7a56f8de3b d29384247babb327b7f3c0e3ddef40d1eff8ae993e6d4d2ba/spacy-2.1.6-cp35-cp35m-manylin ux1_x86_64.whl#sha256=2441ad9e6195669c13f4035dd77c4206414a40ebb8543c69f3386d984a bfb942 (from https://pypi.org/simple/spacy/) (requires-python:>=2.7,!=3.0.*,!=3. 1.*,!=3.2.*,!=3.3.*)
Requirement already satisfied: plac<1.0.0,>=0.9.6 in /usr/local/lib/python3.5/di st-packages (from spacy)
Requirement already satisfied: cymem<2.1.0,>=2.0.2 in /usr/local/lib/python3.5/d ist-packages (from spacy)
Requirement already satisfied: srsly<1.1.0,>=0.0.6 in /usr/local/lib/python3.5/d ist-packages (from spacy)
Requirement already satisfied: blis<0.3.0,>=0.2.2 in /usr/local/lib/python3.5/di st-packages (from spacy)
Requirement already satisfied: requests<3.0.0,>=2.13.0 in /usr/local/lib/python3 .5/dist-packages (from spacy)
Requirement already satisfied: murmurhash<1.1.0,>=0.28.0 in /usr/local/lib/pytho n3.5/dist-packages (from spacy)
Requirement already satisfied: wasabi<1.1.0,>=0.2.0 in /usr/local/lib/python3.5/ dist-packages (from spacy)
Requirement already satisfied: thinc<7.1.0,>=7.0.8 in /usr/local/lib/python3.5/d ist-packages (from spacy)
Requirement already satisfied: preshed<2.1.0,>=2.0.1 in /usr/local/lib/python3.5 /dist-packages (from spacy)
Requirement already satisfied: numpy>=1.15.0 in /usr/local/lib/python3.5/dist-pa ckages (from spacy)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/l ocal/lib/python3.5/dist-packages (from requests<3.0.0,>=2.13.0->spacy)
Requirement already satisfied: idna<2.9,>=2.5 in /usr/local/lib/python3.5/dist-p ackages (from requests<3.0.0,>=2.13.0->spacy)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.5/di st-packages (from requests<3.0.0,>=2.13.0->spacy)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python3.5 /dist-packages (from requests<3.0.0,>=2.13.0->spacy)
Requirement already satisfied: tqdm<5.0.0,>=4.10.0 in /usr/local/lib/python3.5/d ist-packages (from thinc<7.1.0,>=7.0.8->spacy)
Installing collected packages: spacy
KilledStack Overflow用户
发布于 2019-07-19 21:22:41
不确定这是否有效，但您是否尝试过从源代码编译？这个脚本在Spacy's pypi页面。不知道这是否会比常规的pip安装更好地管理RAM约束，但这是我会尝试的事情。
# make sure you are using the latest pip
python -m pip install -U pip
git clone https://github.com/explosion/spaCy
cd spaCy
python -m venv .env
source .env/bin/activate
export PYTHONPATH=`pwd`
pip install -r requirements.txt
python setup.py build_ext --inplace页面原文内容由Stack Overflow提供。腾讯云小微IT领域专用引擎提供翻译支持
原文链接：
https://stackoverflow.com/questions/57058641
复制相关文章
相似问题
腾讯云开发者
