使用异步并行多线程下载-等待
使用异步并行多线程下载-等待
提问于 2018-05-14 12:34:27
我有1000多个大文件可以从我的windows服务下载- C#。其要求是一次维护最多4次并行web文件下载.
我可以使用异步等待实现并发/并行下载，还是必须使用BackgroundWorker进程或线程？异步等待多线程吗？请参阅我使用异步的示例程序-等待如下：
static int i = 0;
Timer_tick() {
while (i < 4) {
i++;
model = GetNextModel();
await Download(model);
}
}
private async Download(XYZ model) {
Task<FilesetResult> t = DoWork(model);
result = await t;
//Use Result
}
private async Task<FilesetResult> Work(XYZ model) {
fileresult = await api.Download(model.path)
i--;
return filesetresult;
}回答 2
Stack Overflow用户
回答已采纳
发布于 2018-05-14 12:42:06
可以使用SemaphoreSlim类限制并行运行的异步任务的数量。类似于：
List<DownloadRequest> requests = Enumerable.Range(0, 100).Select(x => new DownloadRequest()).ToList();
using (var throttler = new SemaphoreSlim(4))
{
Task<DownloadResult>[] downloadTasks = requests.Select(request => Task.Run(async () =>
{
await throttler.WaitAsync();
try
{
return await DownloadTaskAsync(request);
}
finally
{
throttler.Release();
}
})).ToArray();
await Task.WhenAll(downloadTasks);
}更新：谢谢您的评论，解决了问题。
Update2：动态请求列表的示例解决方案
public class DownloadManager : IDisposable
{
private readonly SemaphoreSlim _throttler = new SemaphoreSlim(4);
public async Task<DownloadResult> DownloadAsync(DownloadRequest request)
{
await _throttler.WaitAsync();
try
{
return await api.Download(request);
}
finally
{
_throttler.Release();
}
}
public void Dispose()
{
_throttler?.Dispose();
}
}Stack Overflow用户
发布于 2018-05-14 12:48:46
用手做这件事似乎非常复杂。
var files = new List<Uri>();
Parallel.ForEach(files,
new ParallelOptions { MaxDegreeOfParallelism = 4 },
this.Download);现在，您所需要的只是一个单一的、正常的、同步的方法private void Download(Uri file)，您可以开始了。
如果需要生产者/消费者模式，最简单的版本可能是BlockingCollection。
using System;
using System.Collections.Concurrent;
using System.Threading;
using System.Threading.Tasks;
namespace ConsoleApp11
{
internal class Program
{
internal static void Main()
{
using (var queue = new BlockingCollection<Uri>())
{
// starting the producer task:
Task.Factory.StartNew(() =>
{
for (int i = 0; i < 100; i++)
{
// faking read from message queue... we get a new Uri every 100 ms
queue.Add(new Uri("http://www.example.com/" + i));
Thread.Sleep(100);
}
// just to end this program... you don't need to end this, just listen to your message queue
queue.CompleteAdding();
});
// run the consumers:
Parallel.ForEach(queue.GetConsumingEnumerable(), new ParallelOptions { MaxDegreeOfParallelism = 4 }, Download);
}
}
internal static void Download(Uri uri)
{
// download your file here
Console.WriteLine($"Downloading {uri} [.. ]");
Thread.Sleep(1000);
Console.WriteLine($"Downloading {uri} [..... ]");
Thread.Sleep(1000);
Console.WriteLine($"Downloading {uri} [....... ]");
Thread.Sleep(1000);
Console.WriteLine($"Downloading {uri} [......... ]");
Thread.Sleep(1000);
Console.WriteLine($"Downloading {uri} [..........]");
Thread.Sleep(1000);
Console.WriteLine($"Downloading {uri} OK");
}
}
}页面原文内容由Stack Overflow提供。腾讯云小微IT领域专用引擎提供翻译支持
原文链接：
https://stackoverflow.com/questions/50330471
复制相关文章
相似问题
腾讯云开发者
