blocks|key|474315|text|这种误解可能与Lambda如何处理缩放有关。每当有超过单个Lambda执行上下文/实例所能处理的事件时，Lambda只是创建更多的执行上下文/实例来处理这些事件。|type|unstyled|depth|inlineStyleRanges|entityRanges|data|474316|可能发生的情况是Lambda看到队列中有一堆消息，它试图尽可能快地处理这些消息。它创建了一个Lambda实例来处理第一个事件，然后与SQS对话并请求更多的工作。当它收到下一批消息时，第一个实例仍然很忙，因此它扩展并创建了第二个用于并行的第二批，等等。|offset|length|style|BOLD|474317|这就是你在几分钟内完成象征性预算的原因。|474318|通过使用保留的并发性，可以限制允许Lambda并行执行的函数数量--以下是参考的文档。如果将保留的并发设置为1，则不存在并行化，只允许一个Lambda处理消息。|474319|然而，这将为打开另一个问题--。如果单个Lambda处理消息所花费的时间少于60秒，Lambda将再次使用另一批ASAP调用它，您可能会再次检查您的预算。|474320|在这一点上，一种相对简单的方法是确保lambda函数总是花费大约60秒的时间，在结束时添加一个睡眠时间。|474321|entityMap|0|LINK|mutability|MUTABLE|url|https://docs.aws.amazon.com/lambda/latest/dg/configuration-concurrency.html^0|0|39|4|0|0|14|2|0|0|6|9|0|0^^$0|@$1|2|3|4|5|6|7|X|8|@]|9|@]|A|$]]|$1|B|3|C|5|6|7|Y|8|@$D|Z|E|10|F|G]]|9|@]|A|$]]|$1|H|3|I|5|6|7|11|8|@]|9|@]|A|$]]|$1|J|3|K|5|6|7|12|8|@]|9|@$D|13|E|14|1|15]]|A|$]]|$1|L|3|M|5|6|7|16|8|@$D|17|E|18|F|G]]|9|@]|A|$]]|$1|N|3|O|5|6|7|19|8|@]|9|@]|A|$]]|$1|P|3|-4|5|6|7|1A|8|@]|9|@]|A|$]]]|Q|$R|$5|S|T|U|A|$V|W]]]]

The misunderstanding is probably related to how Lambda handles scaling.
Whenever there are more events than a single Lambda execution context/instance can handle, Lambda just creates more execution contexts/instances to process these events.
What probably happened is that Lambda saw there are a bunch of messages in the queue and it tries to work on these as fast as possible. It created a Lambda instance to handle the first event and then talked to SQS and asked for more work. When it got the next batch of messages, the first instance was still busy, so it scaled out and created a second one that worked on the second batch in parallel, etc. etc.
That's how you ended up going through your token budget in a few minutes.
You can limit how many functions Lambda is allowed to execute in parallel by using reserved concurrency - here are the <a href="https://docs.aws.amazon.com/lambda/latest/dg/configuration-concurrency.html" rel="nofollow noreferrer">docs</a> for reference. If you set the reserved concurrency to 1, there will be no parallelization and only one Lambda is allowed to work on the messages.
This however opens you up to another issue. If that single Lambda takes less than 60 seconds to process the messages, Lambda will call it again with another batch ASAP and you might go over your budget again.
At this point a relatively simple approach would be to make sure that your lambda function always takes about 60 seconds by adding a sleep for the remaining time at the end.

I have 2 Lambda Functions and an SQS queue inbetween.
The first Lambda sends the messages to the Queue.
Then second Lambda has a trigger for this Queue with a batch size of 250 and a batch window of 65 seconds.
I expect the second Lambda to be triggered in batches of 250 messages after about every 65 seconds. In the second Lambda I'm calling a 3rd party API that is limited to 250 API calls per minute (I get 250 tokens per minute).
I tested this setup with for 32.000 messages being added to the queue and the second Lambda didn't pick up the messages in batches as expected. At first it got executed for 15k messages and then there were not enough tokens so it did not process those messages.
The 3rd party API is based on a token bucket with a fill rate of 250 per minute and a maximum capacity of 15.000. It managed to process the first 15.000 messages due to the bucket capacity and then didn't have enough capacity to handle the rest.
I don't understand what went wrong.
<a href="https://i.stack.imgur.com/alA68.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/alA68.png" alt="enter image description here" /></a>

SQS batching for Lambda trigger doesn't work as expected

翻译质量差，导致语言生硬或混乱。

没有提供实际的解决方法或示例。

解答不清晰，无法理解或解决问题。

页面排版不美观，阅读体验差。

文章

问答

视频

学习中心

腾讯云实验室

直播

竞赛

腾讯云代码分析专区

腾讯iOA零信任安全管理系统专区

腾讯云架构师技术同盟交流圈

腾讯云数据库专区

腾讯云顾问专区

腾讯云原生专区

腾讯混元专区

腾讯云TCE专区

腾讯云Lighthouse专区

腾讯云HAI专区

腾讯云Edgeone专区

腾讯云存储专区

腾讯云智能专区

腾讯轻联专区 

腾讯云开发专区

TAPD专区

腾讯轻量云游戏服专区

腾讯云最具价值专家

腾讯云架构师技术同盟

腾讯云创作之星

腾讯云开发者先锋

腾讯云代码助手

云原生构建

TAPD 敏捷项目管理

Cloud Studio

SDK中心

API中心

命令行工具

涵盖代码开发、场景应用、自动测试全流程，助你从零构建专属AI助手

一站式MCP教程库，解锁AI应用新玩法

我有两个Lambda函数和一个SQS队列。第一个Lambda将消息发送到队列。然后，第二个Lambda为这个队列设置了一个触发器，批处理大小为250，批处理窗口为65秒。我预计第二个Lambda将在大约每65秒钟触发一次250条消息。在第二个Lambda中，我调用的第三方API限制在每分钟250个API调用(我每分钟得...

Lambda触发器的SQS批处理不像预期的那样工作-腾讯云开发者社区-腾讯云

问Lambda触发器的SQS批处理不像预期的那样工作
EN

社区

活动

圈层

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问Lambda触发器的SQS批处理不像预期的那样工作EN

社区

活动

圈层

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问Lambda触发器的SQS批处理不像预期的那样工作
EN