如何使用TFF框架进行异步模型训练?
我回顾了迭代训练过程循环,但是我不确定如何知道接收到了哪些客户端模型。
发布于 2020-05-10 21:24:16
在TFF中模拟类似“异步FL”的东西是很有可能的。考虑这一点的一种方法是在概念上将模拟时间与挂钟时间解耦。
每轮对不同数量的客户端进行采样(而不是通常所做的统一K客户端),可能会模拟异步FL。首先只处理选定客户端的一部分是可能的,研究人员可以根据自己的需要自由地对数据/计算进行切片。
Python风格的伪代码演示了这两种技术,不同的客户端采样和延迟的渐变应用:
state = fed_avg_iter_proc.initialize()
for round_num in range(NUM_ROUNDS):
# Here we conceptualize a "round" as a block of time, rather than a synchronous
# round. We have a function that determines which clients will "finish" within
# our configured block of time. This might even return only a single client.
participants = get_next_clients(time_window=timedelta(minutes=30))
num_participants = len(participants)
# Here we only process the first half, and then updated the global model.
state2, metrics = fed_avg_iter_proc.next(state, participants[:num_participants/2])
# Now process the second half of the selected clients.
# Note: this is now apply the 'pseudo-gradient' that was computed on clients
# (the difference between the original `state` and their local training result),
# to the model that has already taken one step (`state2`). This possibly has
# undesirable effects on the optimisation process, or may be improved with
# techniques that handle "stale" gradients.
state3, metrics = fed_avg_iter_proc.next(state2, participants[num_participants/2:])
# Finally update the state for the next for-loop of the simulation.
state = state3https://stackoverflow.com/questions/61152605
复制相似问题