blocks|key|602402|text|根据您的描述，我建议您可以尝试更改设计，以避免有数据开放，因为我们无法节流流分析工作。|type|unstyled|depth|inlineStyleRanges|entityRanges|data|602403|我建议你试着使用流分析窗口函数的翻滚窗。|offset|length|602404|滚动窗口函数用于将数据流分割成不同的时间段，并对它们执行一个函数。|blockquote|602405|我建议您可以使用此函数从一段时间内获取数据。然后，您可以将这些数据合并到一个文档中，并存储到DB中。|602406|例如，可以将窗口设置为1秒。这意味着您在一分钟内将60条记录插入数据库。|602407|entityMap|0|LINK|mutability|MUTABLE|url|https://learn.microsoft.com/en-us/azure/stream-analytics/stream-analytics-window-functions|1|https://msdn.microsoft.com/library/en-us/Dn835055.aspx^0|0|8|7|0|G|3|1|0|0|0|0^^$0|@$1|2|3|4|5|6|7|W|8|@]|9|@]|A|$]]|$1|B|3|C|5|6|7|X|8|@]|9|@$D|Y|E|Z|1|10]|$D|11|E|12|1|13]]|A|$]]|$1|F|3|G|5|H|7|14|8|@]|9|@]|A|$]]|$1|I|3|J|5|6|7|15|8|@]|9|@]|A|$]]|$1|K|3|L|5|6|7|16|8|@]|9|@]|A|$]]|$1|M|3|-4|5|6|7|17|8|@]|9|@]|A|$]]]|N|$O|$5|P|Q|R|A|$S|T]]|U|$5|P|Q|R|A|$S|V]]]]

According to your description, I suggest you could try to change the design to avoid having dataloss, since the we couldn't throttle the Stream Analytics job.

I suggest you could try to use <a href="https://learn.microsoft.com/en-us/azure/stream-analytics/stream-analytics-window-functions" rel="nofollow noreferrer">Stream Analytics Window functions</a>'s <a href="https://msdn.microsoft.com/library/en-us/Dn835055.aspx" rel="nofollow noreferrer">Tumbling Window</a>.

<blockquote>
 Tumbling window functions are used to segment a data stream into distinct time segments and perform a function against them
</blockquote>

I suggest you could use this function to get the data from a period of time. Then you could combine these data into one document and store into the DB.

For example, you could set the windows to 1 seconds. It means you insert 60 records to the db in one minutes.

blocks|key|1942930|text|您为您的收藏准备了多少RU？您是否尝试过简单地扩展您的收藏以满足需求？CosmosDB最近还引入了RU/+can的概念，它可以在背压场景中有所帮助，因为当您超过每秒准备的吞吐量时，它提供了额外的吞吐量。|type|unstyled|depth|inlineStyleRanges|entityRanges|data|1942931|这篇文章很好地介绍了如何使用RU/分钟。|offset|length|1942932|Azure+Dev博客更详细地讨论了这一点。。文章的相关引文(粗体是我的重点)|1942933|为了测试RU/m，我们在开发环境中设计了追赶或故障恢复测试。以前，我们为收集分配了300+for+/s。我们启用了RU/m，并将计算容量从300+and+/秒降至100+and/秒。这给了我们额外的100万RU/m。为了将我们的写操作推向极限，并测试我们的追赶场景，我们模拟了一个上游故障。我们停止播放了大约20个小时。然后，我们开始流待办数据，并观察应用程序是否可以追赶较低的RU/s加上额外的RU/m。开发环境的负载也与我们在生产中看到的相同。|style|BOLD|1942934|entityMap|0|LINK|mutability|MUTABLE|url|https://learn.microsoft.com/en-us/azure/cosmos-db/request-units-per-minute|1|https://azure.microsoft.com/en-us/blog/azure-cosmosdb-case-study-around-ru-min-with-the-universal-store-team/^0|0|0|4|0|0|0|M|1|0|41|1Q|0^^$0|@$1|2|3|4|5|6|7|V|8|@]|9|@]|A|$]]|$1|B|3|C|5|6|7|W|8|@]|9|@$D|X|E|Y|1|Z]]|A|$]]|$1|F|3|G|5|6|7|10|8|@]|9|@$D|11|E|12|1|13]]|A|$]]|$1|H|3|I|5|6|7|14|8|@$D|15|E|16|J|K]]|9|@]|A|$]]|$1|L|3|-4|5|6|7|17|8|@]|9|@]|A|$]]]|M|$N|$5|O|P|Q|A|$R|S]]|T|$5|O|P|Q|A|$R|U]]]]

How many RU have you provisioned for your collection? Have you tried simply scaling your collection to meet the demand? CosmosDB also recently introduced the concept of RU/minute which can help in backpressure scenarios as it provides additional throughput once you've exceeded your provisioned throughput per second. 

<a href="https://learn.microsoft.com/en-us/azure/cosmos-db/request-units-per-minute" rel="nofollow noreferrer">This article</a> provides a good introduction to using RU/minute.

<a href="https://azure.microsoft.com/en-us/blog/azure-cosmosdb-case-study-around-ru-min-with-the-universal-store-team/" rel="nofollow noreferrer">The Azure Dev blog talks about this more in detail</a>. Relevant quote from the article (the bolding is mine for emphasis)

To test RU/m, we designed the catchup or failure recovery test in our dev environment. Previously we allocated 300K RU/s for the collection. We enabled RU/m and reduced compute capacity from 300K RU/second to 100K RU/second. This gave us extra 1M RU/m. To push our writes to the limit and test our catchup scenario we simulated an upstream failure. We stopped streaming for about 20 hours. We then started streaming backlog data and observed if the application could catchup with the lower RU/s plus additional RU/m. The dev environment also had the same load as we see in production.

blocks|key|1900878|text|根据下面的MSDN+ASA实际上在429上重试，我只是期待另一个行为。https://blogs.msdn.microsoft.com/streamanalytics/2016/04/18/troubleshooting-azure-stream-analytics-jobs-on-new-portal-2/|type|unstyled|depth|inlineStyleRanges|entityRanges|offset|length|data|1900879|entityMap|0|LINK|mutability|MUTABLE|url|https://blogs.msdn.microsoft.com/streamanalytics/2016/04/18/troubleshooting-azure-stream-analytics-jobs-on-new-portal-2/^0|Z|3C|0|0^^$0|@$1|2|3|4|5|6|7|L|8|@]|9|@$A|M|B|N|1|O]]|C|$]]|$1|D|3|-4|5|6|7|P|8|@]|9|@]|C|$]]]|E|$F|$5|G|H|I|C|$J|K]]]]

According to the following MSDN ASA does in fact retry on 429, i just expected another behavior.
<a href="https://blogs.msdn.microsoft.com/streamanalytics/2016/04/18/troubleshooting-azure-stream-analytics-jobs-on-new-portal-2/" rel="nofollow noreferrer">https://blogs.msdn.microsoft.com/streamanalytics/2016/04/18/troubleshooting-azure-stream-analytics-jobs-on-new-portal-2/</a>

I have problems that my CosmosDB is busy and therefore i am losing data since the Stream Analyzer isn't buffering data.

My current Azure setup is as follows:

WebAPI -> EventHub -> Stream Analytics job -> CosmosDB

My issue is that if i start my stream back in time and it then has alot of records, the CosmosDB is busy and returns HTTP 429 (Exceeded limit), this will then cause me to have dataloss.

Is there a way to throttle the Stream Analytics job or should i make a completely different design to avoid having dataloss?

Using Stream Analytics to insert data into CosmosDB

翻译质量差，导致语言生硬或混乱。

没有提供实际的解决方法或示例。

解答不清晰，无法理解或解决问题。

页面排版不美观，阅读体验差。

文章

问答

视频

学习中心

腾讯云实验室

直播

竞赛

腾讯云代码分析专区

腾讯iOA零信任安全管理系统专区

腾讯云架构师技术同盟交流圈

腾讯云数据库专区

腾讯云顾问专区

腾讯云原生专区

腾讯混元专区

腾讯云TCE专区

腾讯云Lighthouse专区

腾讯云HAI专区

腾讯云Edgeone专区

腾讯云存储专区

腾讯云智能专区

腾讯轻联专区 

腾讯云开发专区

TAPD专区

腾讯轻量云游戏服专区

腾讯云最具价值专家

腾讯云架构师技术同盟

腾讯云创作之星

腾讯云开发者先锋

腾讯云代码助手

云原生构建

TAPD 敏捷项目管理

Cloud Studio

SDK中心

API中心

命令行工具

涵盖代码开发、场景应用、自动测试全流程，助你从零构建专属AI助手

一站式MCP教程库，解锁AI应用新玩法

我的CosmosDB很忙，所以我失去了数据，因为Stream没有缓冲数据。我目前的Azure设置如下：WebAPI -> EventHub ->流分析工作-> CosmosDB我的问题是，如果我回过头来启动我的流，然后它有很多记录，CosmosDB会很忙，并返回HTTP429(超过限制)，这将导致我拥有dataloss...

问利用流分析将数据插入CosmosDB
EN

Stack Overflow用户

社区

活动

圈层

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问利用流分析将数据插入CosmosDBEN