.net - .Net HttpClient.GetStreamAsync() 的行为与 .GetAsync() 不同

Question

我一直在尝试解决在我的应用程序中下载一批图像时遇到的问题。

如果我HttpClient.GetStreamAsync(url)用来下载批处理，那么似乎某些请求会超时，并最终出错。

但是，如果我使用HttpClient.GetAsync(url)，那么整个批次将毫无问题地下载。

我怀疑它与调用时未释放端口有关.GetStreamAsync(url)，但是我可能在胡说八道。

下面是演示该问题的代码片段。

async Task Main()
{   
    HttpClient httpclient = new HttpClient();
    var imageUrl = "https://tenlives.com.au/wp-content/uploads/2020/09/Found-Kitten-0-8-Weeks-Busy-scaled.jpg";
    var downloadTasks = Enumerable.Range(0, 15)
                                .Select(async u =>
                                {
                                    try
                                    {
                                        //Option 1) - this will fail
                                        var stream = await httpclient.GetStreamAsync(imageUrl);
                                        //End Option 1)

                                        //Option 2) - this will succeed
                                        //var response = await httpclient.GetAsync(imageUrl);
                                        //response.EnsureSuccessStatusCode();
                                        //var stream = await response.Content.ReadAsStreamAsync();
                                        //End Option 2)

                                        return stream;
                                    }
                                    catch (Exception e)
                                    {
                                        Console.WriteLine($"Error downloading image");
                                        throw;
                                    }
                                }).ToList();
    

    try
    {
        await Task.WhenAll(downloadTasks);
    }
    catch (Exception e)
    {       
        Console.WriteLine("================ Failed to download one or more image " + e.Message);
    }
    Console.WriteLine($"Successful downloads: {downloadTasks.Where(t => t.Status == TaskStatus.RanToCompletion).Count()}");
}

在 linqSelect语句的代码块中，Option 1)将如上所述失败。如果您注释掉 1)，并取消注释选项 2)，那么一切都会成功。

谁能解释这里可能发生的事情？

编辑：这似乎适用于 .net 核心。我可以使用 .net framework 4.7.2 及更低版本重现此问题

EDIT2：我还观察到，如果我通过添加来增加默认连接限制， ServicePointManager.DefaultConnectionLimit = 30; 则不再发生错误，但这并不能解释为什么选项 1) 失败但选项 2 成功)

标签： .nethttpclient

Answer 1

正如@RichardDeeming 所解释的HttpClient.GetStreamAsync，HttpClient.GetAsync使用HttpCompletionOption.ResponseHeadersRead.

代码可以改写如下：

async Task Main()
{   
    HttpClient httpclient = new HttpClient();
    var imageUrl = "https://tenlives.com.au/wp-content/uploads/2020/09/Found-Kitten-0-8-Weeks-Busy-scaled.jpg";
    var downloadTasks = Enumerable.Range(0, 15)
        .Select(async u =>
        {
            try
            {
                //Option 1) - this will fail
                var response = await httpclient.GetAsync(imageUrl, HttpCompletionOption.ResponseHeadersRead);
                //End Option 1)

                //Option 2) - this will succeed
                //var response = await httpclient.GetAsync(imageUrl, HttpCompletionOption.ResponseContentRead);
                //End Option 2)

                response.EnsureSuccessStatusCode();
                var stream = await response.Content.ReadAsStreamAsync();
                return stream;
            }
            catch (Exception e)
            {
                Console.WriteLine($"Error downloading image");
                throw;
            }
        }).ToList();

    try
    {
        await Task.WhenAll(downloadTasks);
    }
    catch (Exception e)
    {       
        Console.WriteLine("================ Failed to download one or more image " + e.Message);
    }
    Console.WriteLine($"Successful downloads: {downloadTasks.Where(t => t.Status == TaskStatus.RanToCompletion).Count()}");
}

下一个HttpClient.GetAsync电话HttpClient.SendAsync。我们可以在GitHub 上查看该方法的代码：

//I removed the uninterested code for the question
public Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationToken cancellationToken)
{
    TaskCompletionSource<HttpResponseMessage> tcs = new TaskCompletionSource<HttpResponseMessage>();
    client.SendAsync(request, cancellationToken).ContinueWith(task =>
    {
        HttpResponseMessage response = task.Result;
        if(completionOption == HttpCompletionOption.ResponseHeadersRead)
        {
            tcs.TrySetResult(response);
        }
        else
        {
            response.Content.LoadIntoBufferAsync(int.MaxValue).ContinueWith(contentTask =>
            {
                tcs.TrySetResult(response);
            });
        }
    });
    return tcs.Task;
}

使用HttpClient.GetAsync（或SendAsync(HttpCompletionOption.ResponseContentRead)），网络缓冲区中的接收内容被读取并集成到本地缓冲区中。

我不确定网络缓冲区，但我认为某处的缓冲区（网卡、操作系统、HttpClient、???）已满并阻止新响应。

您可以通过正确管理此缓冲区来更正代码，例如通过处理关联的流：

var downloadTasks = Enumerable.Range(0, 15)
.Select(async u =>
{
    try
    {
        var stream = await httpclient.GetStreamAsync(imageUrl);
        stream.Dispose(); //Free buffer
        return stream;
    }
    catch (Exception e)
    {
        Console.WriteLine($"Error downloading image");
        throw;
    }
}).ToList();

在 .Net Core 中，原始代码无需更正即可工作。HttpClient 类已被重写，当然也得到了改进。