Asyncio.gather vs asyncio.wait
Asyncio.gather vs asyncio.wait
提问于 2017-02-15 00:21:48
asyncio.gather和asyncio.wait似乎有相似的用途:我有一堆想要执行/等待的异步事务(不一定要等到一个完成后才开始下一个)。它们使用不同的语法，在一些细节上也不同，但在我看来，有两个函数在功能上有如此巨大的重叠，这对我来说是非常不自然的。我遗漏了什么？
回答 4
Stack Overflow用户
回答已采纳
发布于 2017-02-15 18:24:10
虽然在一般情况下类似(“运行并获取多个任务的结果”)，但每个函数在其他情况下都有一些特定的功能：
返回Future实例，允许对任务进行高级分组：
import asyncio
from pprint import pprint
import random
async def coro(tag):
print(">", tag)
await asyncio.sleep(random.uniform(1, 3))
print("<", tag)
return tag
loop = asyncio.get_event_loop()
group1 = asyncio.gather(*[coro("group 1.{}".format(i)) for i in range(1, 6)])
group2 = asyncio.gather(*[coro("group 2.{}".format(i)) for i in range(1, 4)])
group3 = asyncio.gather(*[coro("group 3.{}".format(i)) for i in range(1, 10)])
all_groups = asyncio.gather(group1, group2, group3)
results = loop.run_until_complete(all_groups)
loop.close()
pprint(results)可以通过调用group2.cancel()甚至all_groups.cancel()来取消组中的所有任务。另请参阅.gather(..., return_exceptions=True)，
支持在第一个任务完成后等待停止，或在指定的超时后等待停止，允许较低的操作精度：
import asyncio
import random
async def coro(tag):
print(">", tag)
await asyncio.sleep(random.uniform(0.5, 5))
print("<", tag)
return tag
loop = asyncio.get_event_loop()
tasks = [coro(i) for i in range(1, 11)]
print("Get first result:")
finished, unfinished = loop.run_until_complete(
asyncio.wait(tasks, return_when=asyncio.FIRST_COMPLETED))
for task in finished:
print(task.result())
print("unfinished:", len(unfinished))
print("Get more results in 2 seconds:")
finished2, unfinished2 = loop.run_until_complete(
asyncio.wait(unfinished, timeout=2))
for task in finished2:
print(task.result())
print("unfinished2:", len(unfinished2))
print("Get all other results:")
finished3, unfinished3 = loop.run_until_complete(asyncio.wait(unfinished2))
for task in finished3:
print(task.result())
loop.close()Stack Overflow用户
发布于 2017-06-18 18:40:32
与asyncio.gather相比，asyncio.wait更底层。
顾名思义，asyncio.gather主要关注于收集结果。它等待一堆期货，并按给定的顺序返回结果。
asyncio.wait只是在等待未来。它不是直接给你结果，而是给出已完成和未决的任务。您必须手动收集这些值。
此外，您可以指定等待所有期货都完成，或者只等待第一个包含wait的期货。
Stack Overflow用户
发布于 2020-08-23 17:01:44
当涉及到异常时，这两个函数的默认行为是很容易被忽略的一个非常重要的区别。
我将使用这个例子来模拟一个协程，它会引发异常，有时-
import asyncio
import random
async def a_flaky_tsk(i):
await asyncio.sleep(i) # bit of fuzz to simulate a real-world example
if i % 2 == 0:
print(i, "ok")
else:
print(i, "crashed!")
raise ValueError
coros = [a_flaky_tsk(i) for i in range(10)]await asyncio.gather(*coros)输出-
0 ok
1 crashed!
Traceback (most recent call last):
File "/Users/dev/PycharmProjects/trading/xxx.py", line 20, in <module>
asyncio.run(main())
File "/Users/dev/.pyenv/versions/3.8.2/lib/python3.8/asyncio/runners.py", line 43, in run
return loop.run_until_complete(main)
File "/Users/dev/.pyenv/versions/3.8.2/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
return future.result()
File "/Users/dev/PycharmProjects/trading/xxx.py", line 17, in main
await asyncio.gather(*coros)
File "/Users/dev/PycharmProjects/trading/xxx.py", line 12, in a_flaky_tsk
raise ValueError
ValueError如您所见，索引1之后的coros从未执行过。
但await asyncio.wait(coros)会继续执行任务，即使其中一些失败了-
0 ok
1 crashed!
2 ok
3 crashed!
4 ok
5 crashed!
6 ok
7 crashed!
8 ok
9 crashed!
Task exception was never retrieved
future: <Task finished name='Task-10' coro=<a_flaky_tsk() done, defined at /Users/dev/PycharmProjects/trading/xxx.py:6> exception=ValueError()>
Traceback (most recent call last):
File "/Users/dev/PycharmProjects/trading/xxx.py", line 12, in a_flaky_tsk
raise ValueError
ValueError
Task exception was never retrieved
future: <Task finished name='Task-8' coro=<a_flaky_tsk() done, defined at /Users/dev/PycharmProjects/trading/xxx.py:6> exception=ValueError()>
Traceback (most recent call last):
File "/Users/dev/PycharmProjects/trading/xxx.py", line 12, in a_flaky_tsk
raise ValueError
ValueError
Task exception was never retrieved
future: <Task finished name='Task-2' coro=<a_flaky_tsk() done, defined at /Users/dev/PycharmProjects/trading/xxx.py:6> exception=ValueError()>
Traceback (most recent call last):
File "/Users/dev/PycharmProjects/trading/xxx.py", line 12, in a_flaky_tsk
raise ValueError
ValueError
Task exception was never retrieved
future: <Task finished name='Task-9' coro=<a_flaky_tsk() done, defined at /Users/dev/PycharmProjects/trading/xxx.py:6> exception=ValueError()>
Traceback (most recent call last):
File "/Users/dev/PycharmProjects/trading/xxx.py", line 12, in a_flaky_tsk
raise ValueError
ValueError
Task exception was never retrieved
future: <Task finished name='Task-3' coro=<a_flaky_tsk() done, defined at /Users/dev/PycharmProjects/trading/xxx.py:6> exception=ValueError()>
Traceback (most recent call last):
File "/Users/dev/PycharmProjects/trading/xxx.py", line 12, in a_flaky_tsk
raise ValueError
ValueError当然，这两种行为都可以通过使用-
asyncio.gather(..., return_exceptions=True)
或,
asyncio.wait([...], return_when=asyncio.FIRST_EXCEPTION)
但它不会就此结束！
注意:上面日志中的Task exception was never retrieved。
在您逐个await子任务之前，asyncio.wait()不会从子任务中重新引发异常。(日志中的堆栈跟踪只是消息，它们不能被捕获！)
done, pending = await asyncio.wait(coros)
for tsk in done:
try:
await tsk
except Exception as e:
print("I caught:", repr(e))输出-
0 ok
1 crashed!
2 ok
3 crashed!
4 ok
5 crashed!
6 ok
7 crashed!
8 ok
9 crashed!
I caught: ValueError()
I caught: ValueError()
I caught: ValueError()
I caught: ValueError()
I caught: ValueError()另一方面，要用asyncio.gather()捕获异常，您必须-
results = await asyncio.gather(*coros, return_exceptions=True)
for result_or_exc in results:
if isinstance(result_or_exc, Exception):
print("I caught:", repr(result_or_exc))(输出与之前相同)
页面原文内容由Stack Overflow提供。腾讯云小微IT领域专用引擎提供翻译支持
原文链接：
https://stackoverflow.com/questions/42231161
复制相关文章
相似问题