blocks|key|1534468|text|似乎很少在子进程中使用Nltk和Python请求。尝试使用线程而不是进程，我与其他库和请求有完全相同的问题，用线程替换进程对我很有帮助。|type|unstyled|depth|inlineStyleRanges|entityRanges|data|1534469|entityMap^0|0^^$0|@$1|2|3|4|5|6|7|D|8|@]|9|@]|A|$]]|$1|B|3|-4|5|6|7|E|8|@]|9|@]|A|$]]]|C|$]]

It seems using Nltk and Python Requests in a child process is rare. Try using Thread instead of Process, I was having exactly same issue with some other library and Requests and replacing Process with Thread worked for me.

blocks|key|1817700|text|更新python库和python应该可以解决这个问题：|type|unstyled|depth|inlineStyleRanges|entityRanges|data|1817701|alvas@ubi:~$+pip+freeze+%7C+grep+nltk
nltk==3.0.3
alvas@ubi:~$+pip+freeze+%7C+grep+requests
requests==2.7.0
alvas@ubi:~$+python+--version
Python+2.7.6
alvas@ubi:~$+lsb_release+-a
No+LSB+modules+are+available.
Distributor+ID:+Ubuntu
Description:++++Ubuntu+14.04.2+LTS
Release:++++14.04
Codename:+++trusty|code-block|syntax|javascript|1817702|来自代码：|1817703|from+multiprocessing+import+Process
import+nltk
import+time


def+child_fn():
++++print+"Fetch+URL"
++++import+urllib2
++++print+urllib2.urlopen("https://www.google.com").read()[:100]
++++print+"Done"


while+True:
++++child_process+=+Process(target=child_fn)
++++child_process.start()
++++child_process.join()
++++print+"Child+process+returned"
++++time.sleep(1)|1817704|输出|1817705|Fetch+URL
<!doctype+html><html+itemscope=""+itemtype="http://schema.org/WebPage"+lang="de"><head><meta+content
Done
Child+process+returned
Fetch+URL
<!doctype+html><html+itemscope=""+itemtype="http://schema.org/WebPage"+lang="de"><head><meta+content
Done
Child+process+returned
Fetch+URL
<!doctype+html><html+itemscope=""+itemtype="http://schema.org/WebPage"+lang="de"><head><meta+content
Done
Child+process+returned|1817706|1817707|alvas@ubi:~$+python
Python+2.7.6+(default,+Jun+22+2015,+17:58:13)+
[GCC+4.8.2]+on+linux2
Type+"help",+"copyright",+"credits"+or+"license"+for+more+information.
>>>+from+multiprocessing+import+Process
>>>+import+requests
>>>+from+pprint+import+pprint
>>>+Process(target=lambda:+pprint(
...+++++++++requests.get('https://api.github.com'))).start()
>>>+<Response+[200]>

>>>+import+nltk
>>>+Process(target=lambda:+pprint(
...+++++++++requests.get('https://api.github.com'))).start()
>>>+<Response+[200]>|1817708|它也应该与python3一起工作：|offset|length|style|CODE|1817709|alvas@ubi:~$+python3
Python+3.4.0+(default,+Jun+19+2015,+14:20:21)+
[GCC+4.8.2]+on+linux
Type+"help",+"copyright",+"credits"+or+"license"+for+more+information.
>>>+from+multiprocessing+import+Process
>>>+import+requests
>>>+Process(target=lambda:+print(requests.get('https://api.github.com'))).start()
>>>+
>>>+<Response+[200]>

>>>+import+nltk
>>>+Process(target=lambda:+print(requests.get('https://api.github.com'))).start()
>>>+<Response+[200]>|1817710|entityMap^0|0|0|0|0|0|0|0|0|5|7|0|0^^$0|@$1|2|3|4|5|6|7|11|8|@]|9|@]|A|$]]|$1|B|3|C|5|D|7|12|8|@]|9|@]|A|$E|F]]|$1|G|3|H|5|6|7|13|8|@]|9|@]|A|$]]|$1|I|3|J|5|D|7|14|8|@]|9|@]|A|$E|F]]|$1|K|3|L|5|6|7|15|8|@]|9|@]|A|$]]|$1|M|3|N|5|D|7|16|8|@]|9|@]|A|$E|F]]|$1|O|3|H|5|6|7|17|8|@]|9|@]|A|$]]|$1|P|3|Q|5|D|7|18|8|@]|9|@]|A|$E|F]]|$1|R|3|S|5|6|7|19|8|@$T|1A|U|1B|V|W]]|9|@]|A|$]]|$1|X|3|Y|5|D|7|1C|8|@]|9|@]|A|$E|F]]|$1|Z|3|-4|5|6|7|1D|8|@]|9|@]|A|$]]]|10|$]]

Updating your python libraries and python should resolve the problem:

<pre><code>alvas@ubi:~$ pip freeze | grep nltk
nltk==3.0.3
alvas@ubi:~$ pip freeze | grep requests
requests==2.7.0
alvas@ubi:~$ python --version
Python 2.7.6
alvas@ubi:~$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 14.04.2 LTS
Release: 14.04
Codename: trusty
</code></pre>

From code:

<pre><code>from multiprocessing import Process
import nltk
import time


def child_fn():
 print "Fetch URL"
 import urllib2
 print urllib2.urlopen("https://www.google.com").read()[:100]
 print "Done"


while True:
 child_process = Process(target=child_fn)
 child_process.start()
 child_process.join()
 print "Child process returned"
 time.sleep(1)
</code></pre>

[out]:

<pre><code>Fetch URL
&lt;!doctype html&gt;&lt;html itemscope="" itemtype="http://schema.org/WebPage" lang="de"&gt;&lt;head&gt;&lt;meta content
Done
Child process returned
Fetch URL
&lt;!doctype html&gt;&lt;html itemscope="" itemtype="http://schema.org/WebPage" lang="de"&gt;&lt;head&gt;&lt;meta content
Done
Child process returned
Fetch URL
&lt;!doctype html&gt;&lt;html itemscope="" itemtype="http://schema.org/WebPage" lang="de"&gt;&lt;head&gt;&lt;meta content
Done
Child process returned
</code></pre>

From code:

<pre><code>alvas@ubi:~$ python
Python 2.7.6 (default, Jun 22 2015, 17:58:13) 
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
&gt;&gt;&gt; from multiprocessing import Process
&gt;&gt;&gt; import requests
&gt;&gt;&gt; from pprint import pprint
&gt;&gt;&gt; Process(target=lambda: pprint(
... requests.get('https://api.github.com'))).start()
&gt;&gt;&gt; &lt;Response [200]&gt;

&gt;&gt;&gt; import nltk
&gt;&gt;&gt; Process(target=lambda: pprint(
... requests.get('https://api.github.com'))).start()
&gt;&gt;&gt; &lt;Response [200]&gt;
</code></pre>

<hr>

It should work with <code>python3</code> too:

<pre><code>alvas@ubi:~$ python3
Python 3.4.0 (default, Jun 19 2015, 14:20:21) 
[GCC 4.8.2] on linux
Type "help", "copyright", "credits" or "license" for more information.
&gt;&gt;&gt; from multiprocessing import Process
&gt;&gt;&gt; import requests
&gt;&gt;&gt; Process(target=lambda: print(requests.get('https://api.github.com'))).start()
&gt;&gt;&gt; 
&gt;&gt;&gt; &lt;Response [200]&gt;

&gt;&gt;&gt; import nltk
&gt;&gt;&gt; Process(target=lambda: print(requests.get('https://api.github.com'))).start()
&gt;&gt;&gt; &lt;Response [200]&gt;
</code></pre>

blocks|key|1476295|text|这个问题似乎还没有解决。https://github.com/nltk/nltk/issues/947我认为这是一个严重的问题(除非你在玩NLTK，做POCs和试用模型，而不是实际的应用程序)，我正在RQ+(http://python-rq.org/)中运行NLP管道。|type|unstyled|depth|inlineStyleRanges|entityRanges|offset|length|data|1476296|nltk==3.2.1
requests==2.9.1|code-block|syntax|javascript|1476297|entityMap|0|LINK|mutability|MUTABLE|url|https://github.com/nltk/nltk/issues/947|1|http://python-rq.org/^0|C|13|0|2W|L|1|0|0^^$0|@$1|2|3|4|5|6|7|S|8|@]|9|@$A|T|B|U|1|V]|$A|W|B|X|1|Y]]|C|$]]|$1|D|3|E|5|F|7|Z|8|@]|9|@]|C|$G|H]]|$1|I|3|-4|5|6|7|10|8|@]|9|@]|C|$]]]|J|$K|$5|L|M|N|C|$O|P]]|Q|$5|L|M|N|C|$O|R]]]]

This issue still seems not solved.
<a href="https://github.com/nltk/nltk/issues/947" rel="nofollow">https://github.com/nltk/nltk/issues/947</a>
I think this is a serious issue (unless you are playing with NLTK, doing POCs and trying out models, not actual apps) 
I am running the NLP pipelines in RQ workers (<a href="http://python-rq.org/" rel="nofollow">http://python-rq.org/</a>)

<pre><code>nltk==3.2.1
requests==2.9.1
</code></pre>

I'm running into an issue when combining multiprocessing, requests (or urllib2) and nltk. Here is a very simple code:

<pre><code>&gt;&gt;&gt; from multiprocessing import Process
&gt;&gt;&gt; import requests
&gt;&gt;&gt; from pprint import pprint
&gt;&gt;&gt; Process(target=lambda: pprint(
 requests.get('https://api.github.com'))).start()
&gt;&gt;&gt; &lt;Response [200]&gt; # this is the response displayed by the call to `pprint`.
</code></pre>

A bit more details on what this piece of code does:

<ol>
<li>Import a few required modules</li>
<li>Start a child process</li>
<li>Issue an HTTP GET request to 'api.github.com' from the child process</li>
<li>Display the result</li>
</ol>

This is working great. The problem comes when importing nltk:

<pre><code>&gt;&gt;&gt; import nltk
&gt;&gt;&gt; Process(target=lambda: pprint(
 requests.get('https://api.github.com'))).start()
&gt;&gt;&gt; # nothing happens!
</code></pre>

After having imported NLTK, the requests actually silently crashes the thread (if you try with a named function instead of the lambda function, adding a few <code>print</code> statement before and after the call, you'll see that the execution stops right on the call to <code>requests.get</code>)
Does anybody have any idea what in NLTK could explain such behavior, and how to get overcome the issue?

Here are the version I'm using:

<pre><code>$&gt; python --version
Python 2.7.5
$&gt; pip freeze | grep nltk
nltk==2.0.5
$&gt; pip freeze | grep requests
requests==2.2.1
</code></pre>

I'm running Mac OS X v. 10.9.5.

Thanks!

Python child process silently crashes when issuing an HTTP request

翻译质量差，导致语言生硬或混乱。

没有提供实际的解决方法或示例。

解答不清晰，无法理解或解决问题。

页面排版不美观，阅读体验差。

文章

问答

视频

学习中心

腾讯云实验室

直播

竞赛

腾讯云代码分析专区

腾讯iOA零信任安全管理系统专区

腾讯云架构师技术同盟交流圈

腾讯云数据库专区

腾讯云顾问专区

腾讯云原生专区

腾讯混元专区

腾讯云TCE专区

腾讯云Lighthouse专区

腾讯云HAI专区

腾讯云Edgeone专区

腾讯云存储专区

腾讯云智能专区

腾讯轻联专区 

腾讯云开发专区

TAPD专区

腾讯轻量云游戏服专区

腾讯云最具价值专家

腾讯云架构师技术同盟

腾讯云创作之星

腾讯云开发者先锋

腾讯云代码助手

云原生构建

TAPD 敏捷项目管理

Cloud Studio

SDK中心

API中心

命令行工具

涵盖代码开发、场景应用、自动测试全流程，助你从零构建专属AI助手

一站式MCP教程库，解锁AI应用新玩法

在合并多处理、请求(或urllib2)和nltk时，我遇到了一个问题。下面是一个非常简单的代码：>>> from multiprocessing import Process>>> import requests>>> from pprint import pprint>>> Process(target=lambda...

问Python子进程在发出HTTP请求时静默崩溃
EN

Stack Overflow用户

社区

活动

圈层

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问Python子进程在发出HTTP请求时静默崩溃EN