我应该使用事件、信号量、锁、条件或它们的组合来管理多线程Python程序的安全退出吗？
我应该使用事件、信号量、锁、条件或它们的组合来管理多线程Python程序的安全退出吗？
提问于 2018-06-07 07:53:52
我正在编写一个多线程python程序，在这个程序中，主线程和它产生的其他线程作为守护进程(但不是Thread.daemon=True)运行，这些守护进程在特定目录中查找特定文件，并在它们存在时对它们执行操作。有可能在一个/任何一个线程中发生错误，这将要求整个程序退出。但是，我需要其他线程在退出之前完成它们当前的工作。
据我所知，如果我为我的衍生线程设置了myThread.daemon=True，它们将在主线程退出时立即自动退出。然而，我希望其他线程在退出之前完成它们当前的工作(除非错误是某种灾难性的失败，在这种情况下，我可能无论如何都会退出所有东西，无论安全与否)。因此，我不会将线程的daemon属性设置为True。
查看线程模块文档和各种可用的对象(如事件、信号量、条件和锁)时，我不确定处理这种情况的最佳方法。此外，当程序由于SIGTERM/SIGINT信号而需要终止时，我不确定如何处理这种情况。
一些代码说明了我的程序结构的一个简化版本：
import threading
import signals
import glob
import time
class MyThread1( threading.thread ):
def __init__( self, name='MyThread1' ):
threading.Thread.__init__( self )
self.name = name
return
def run( self ):
while True:
filePathList = glob.glob( thisThreadDir + '/*.txt' )
for file in filePathList:
try:
doSomeProcessing( file )
# Then move the file to another thread's dir
# or potentially create a new file that will
# be picked up by another thread
except:
# Need to potentially tell all other threads
# to finish task and exit depending on error
# I assume this would be the place to check for some kind of
# flag or other indication to terminate the thread?
time.sleep( 30 )
# Now imagine a few more custom threads with the same basic structure,
# except that what is happening in doSomeProcessing() will obviously vary
# Main Thread/Script
def sigintHandler( SIGINT, frame ):
# How do I handle telling all threads to finish their current loop
# and then exit safely when I encounter this signal?
sys.exit( 1 )
def sigtermHandler( SIGTERM, frame ):
# Same question for this signal handler
sys.exit( 1 )
signal.signal( signal.SIGINT, sigintHandler )
signal.signal( signal.SIGTERM, sigtermHandler )
myOtherThread1 = MyThread1()
myOtherThreadN = MyThreadN()
myOtherThread1.start()
myOtherThreadN.start()
while True:
filePathList = glob.glob( mainDir + '/*.txt' )
for file in filePathList:
try:
doMainProcessing( file )
# Move file or write a new one in another thread's dir
except:
# Again, potentially need to exit the whole program, but want
# the other threads to finish their current loop first
# Check if another thread told us we need to exit?
time.sleep( 30 )回答 1
Stack Overflow用户
回答已采纳
发布于 2018-06-07 08:41:58
我将使用Event 向线程发出退出的信号
- 在
run()__init__ - use
sleep中为事件的wait()创建事件，并检查何时退出 - 从外部设置事件以停止线程
要处理线程中的异常，我会在它所做的一切周围使用一个try/ except块。当捕获到异常时，存储异常(和/或任何其他您需要的信息)，清理并退出线程。
在外部，在主线程中，检查所有线程中的存储异常，如果发现任何异常，则通知所有线程它们应该退出。
在主线程(也包括SIGINT)中处理异常的，在那里有一个try/except块，并向所有线程发出停止的信号。
总而言之，使用虚拟异常和调试打印：
import threading
import time
class MyThread(threading.Thread):
def __init__(self):
super().__init__()
self.stop_requested = threading.Event()
self.exception = None
def run(self):
try:
# sleep for 1 second, or until stop is requested, stop if stop is requested
while not self.stop_requested.wait(1):
# do your thread thing here
print('in thread {}'.format(self))
# simulate a random exception:
import random
if random.randint(0, 50) == 42:
1 / 0
except Exception as e:
self.exception = e
# clean up here
print('clean up thread {}'.format(self))
def stop(self):
# set the event to signal stop
self.stop_requested.set()
# create and start some threads
threads = [MyThread(), MyThread(), MyThread(), MyThread()]
for t in threads:
t.start()
# main thread looks at the status of all threads
try:
while True:
for t in threads:
if t.exception:
# there was an error in a thread - raise it in main thread too
# this will stop the loop
raise t.exception
time.sleep(0.2)
except Exception as e:
# handle exceptions any way you like, or don't
# This includes exceptions in main thread as well as those in other threads
# (because of "raise t.exception" above)
print(e)
finally:
print('clan up everything')
for t in threads:
# threads will know how to clean up when stopped
t.stop()页面原文内容由Stack Overflow提供。腾讯云小微IT领域专用引擎提供翻译支持
原文链接：
https://stackoverflow.com/questions/50731203
复制相关文章
相似问题