python学习笔记10.4 通过killpg杀死进程组

锦小年

发布于 2021-12-08 11:52:57

7600

发布于 2021-12-08 11:52:57

在实际使用多进程的过程中，希望一个子进程报错，则停止所有进程，并退出主进程。在子进程中不能使用exit()函数，它会使得主进程一直等待，程序就卡在这里了。

from  multiprocessing import Pool
import time
import os
import sys
import signal
import subprocess
import platform
def f(i):
    print(f'running {i}, pid: {os.getgid()}')
    if i == 3:
        time.sleep(5)
        print('exit in subp')
        exit()
    time.sleep(3)
    return i

def call_back(i):
    print('callback:',i)

def error_call_back(i):
    print('error:',i)
    
def main():
    mp = Pool(4)
    for i in range(10):
        mp.apply_async(f,args=(i,),callback=call_back,error_callback=error_call_back)
    mp.close()
    mp.join()
if __name__ == '__main__':
    main()

程序运行：

running 0, pid: 1000
running 1, pid: 1000
running 2, pid: 1000
running 3, pid: 1000
callback: 0
running 4, pid: 1000
callback: 1
running 5, pid: 1000
callback: 2
running 6, pid: 1000
exit in subp
None
running 7, pid: 1000
callback: 4
running 8, pid: 1000
callback: 5
running 9, pid: 1000
callback: 6
callback: 7
callback: 8
callback: 9

程序并不会退出。

解决方案：使用在error_callback中使用killpg函数，杀死进程组。

from  multiprocessing import Pool
import time
import os
import sys
import signal
import subprocess
import platform
def f(i):
    print(f'running {i}, pid: {os.getgid()}')
    
    if i == 3:
        time.sleep(5)
        print('subprocess raise error, jump to error callback')
        raise ValueError(f'error in {i}')
    time.sleep(3)
    return i

def call_back(i):
    print('callback:',i)

def error_call_back(i):
    print('error:',i)
    print('exit by killpg')
    kill_sig = signal.SIGKILL
    os_name = platform.system()
    if os_name == 'Windows':
        kill_sig = signal.CTRL_C_EVENT
    os.killpg(os.getpgid(os.getpid()), kill_sig)
    
def main():
    mp = Pool(4)
    for i in range(10):
        mp.apply_async(f,args=(i,),callback=call_back,error_callback=error_call_back)
    mp.close()
    mp.join()
if __name__ == '__main__':
    main()

运行结果:

running 0, pid: 1000
running 1, pid: 1000
running 2, pid: 1000
running 3, pid: 1000
callback: 0
running 4, pid: 1000
callback: 1
running 5, pid: 1000
callback: 2
running 6, pid: 1000
subprocess raise error, jump to error callback
running 7, pid: 1000
error: error in 3
exit by killpg

这样就能解决两个问题：（1）子进程发生异常时所有子进程退出且主进程退出。（2）子进程中不能使用exit()函数问题。

本文参与腾讯云自媒体分享计划，分享自作者个人站点/博客。

原始发表：2021-09-17 ，如有侵权请联系 cloudcommunity@tencent.com 删除

callback