文档章节

C# HttpWebRequest 上传大文件 超时问题

baiwengong
 baiwengong
发布于 2016/05/23 16:57
字数 837
阅读 254
收藏 0

If you ever had to upload large volumes of data over HTTP, you probably ran into timeout issues. The default Timeout value for HttpWebRequest is 100 seconds, which means that if it takes more than that from the time you send the request headers to the time you receive the response headers, your request will fail. Obviously, if you’re uploading a large file, you need to increase that timeout… but to which value?

If you know the available bandwidth, you could calculate a rough estimate of how long it should take to upload the file, but it’s not very reliable, because if there is some network congestion, it will take longer, and your request will fail even though it could have succeeded given enough time. So, should you set the timeout to a very large value, like several hours, or even Timeout.Infinite? Probably not. The most compelling reason is that even though the transfer itself could take hours, some phases of the exchange shouldn’t take that long. Let’s decompose the phases of an HTTP upload:

输入图片说明

Obtaining the request stream or getting the response (orange parts) isn’t supposed to take very long, so obviously we need a rather short timeout there (the default value of 100 seconds seems reasonable). But sending the request body (blue part) could take much longer, and there is no reliable way to decide how long that should be; as long as we keep sending data and the server is receiving it, there is no reason not to continue, even if it’s taking hours. So we actually don’t want a timeout at all there! Unfortunately, the behavior of the Timeout property is to consider everything from the call to GetRequestStream to the return of GetResponse…

In my opinion, it’s a design flaw of the HttpWebRequest class, and one that has bothered me for a very long time. So I eventually came up with a solution. It relies on the fact that the asynchronous versions of GetRequestStream and GetResponse don’t have a timeout mechanism. Here’s what the documentation says:

The Timeout property has no effect on asynchronous requests made with the BeginGetResponse or BeginGetRequestStream method.

In the case of asynchronous requests, the client application implements its own time-out mechanism. Refer to the example in the BeginGetResponse method.

So, a solution could be to to use these methods directly (or the new Task-based versions: GetRequestStreamAsync and GetResponseAsync); but more often than not, you already have an existing code base that uses the synchronous methods, and changing the code to make it fully asynchronous is usually not trivial. So, the easy approach is to create synchronous wrappers around BeginGetRequestStream and BeginGetResponse, with a way to specify a timeout for these operations:

public static class WebRequestExtensions
{
    public static Stream GetRequestStreamWithTimeout(
        this WebRequest request,
        int? millisecondsTimeout = null)
    {
        return AsyncToSyncWithTimeout(
            request.BeginGetRequestStream,
            request.EndGetRequestStream,
            millisecondsTimeout ?? request.Timeout);
    }

    public static WebResponse GetResponseWithTimeout(
        this HttpWebRequest request,
        int? millisecondsTimeout = null)
    {
        return AsyncToSyncWithTimeout(
            request.BeginGetResponse,
            request.EndGetResponse,
            millisecondsTimeout ?? request.Timeout);
    }

    private static T AsyncToSyncWithTimeout<T>(
        Func<AsyncCallback, object, IAsyncResult> begin,
        Func<IAsyncResult, T> end,
        int millisecondsTimeout)
    {
        var iar = begin(null, null);
        if (!iar.AsyncWaitHandle.WaitOne(millisecondsTimeout))
        {
            var ex = new TimeoutException();
            throw new WebException(ex.Message, ex, WebExceptionStatus.Timeout, null);
        }
        return end(iar);
    }
}

(note that I used the Begin/End methods rather than the Async methods, in order to keep compatibility with older versions of .NET)

These extension methods can be used instead of GetRequestStream and GetResponse; each of them will timeout if they take too long, but once you have the request stream, you can take as long as you want to upload the data. Note that the stream itself has its own read and write timeout (5 minutes by default), so if 5 minutes go by without any data being uploaded, the Write method will cause an exception. Here is the new upload scenario using these methods:

timeout2

As you can see, the only difference is that the timeout doesn’t apply anymore to the transfer of the request body, but only to obtaining the request stream and getting the response. Here’s a full example that corresponds to the scenario above:

long UploadFile(string path, string url, string contentType)
{
    // Build request
    var request = (HttpWebRequest)WebRequest.Create(url);
    request.Method = WebRequestMethods.Http.Post;
    request.AllowWriteStreamBuffering = false;
    request.ContentType = contentType;
    string fileName = Path.GetFileName(path);
    request.Headers["Content-Disposition"] = string.Format("attachment; filename=\"{0}\"", fileName);

    try
    {
        // Open source file
        using (var fileStream = File.OpenRead(path))
        {
            // Set content length based on source file length
            request.ContentLength = fileStream.Length;

            // Get the request stream with the default timeout
            using (var requestStream = WebRequestExtensions.GetRequestStreamWithTimeout(request,30000))
            {
                // Upload the file with no timeout
                fileStream.CopyTo(requestStream);
            }
        }

        // Get response with the default timeout, and parse the response body
        using (var response = WebRequestExtensions.GetResponseWithTimeout(request,30000))
        using (var responseStream = response.GetResponseStream())
        using (var reader = new StreamReader(responseStream))
        {
            string json = reader.ReadToEnd();
            var j = JObject.Parse(json);
            return j.Value<long>("Id");
        }
    }
    catch (WebException ex)
    {
        if (ex.Status == WebExceptionStatus.Timeout)
        {
            LogError(ex, "Timeout while uploading '{0}'", fileName);
        }
        else
        {
            LogError(ex, "Error while uploading '{0}'", fileName);
        }
        throw;
    }
}

I hope you will find this helpful!

本文转载自:http://www.thomaslevesque.com/2014/01/14/tackling-timeout-issues-when-uploading-large-files-with-...

baiwengong
粉丝 0
博文 47
码字总数 5483
作品 0
宝山
私信 提问
HttpWebRequest 基础连接已经关闭: 接收时发生错误

HttpWebRequest 基础连接已经关闭: 接收时发生错误 HttpWebRequest 基础连接已经关闭: 发送时发生错误 HttpWebRequest 基础连接已关闭 连接意外关闭 1.发生问题场景描述(可以直接跳过,不看...

sddhome
01/17
0
0
C# WinForm开发系列 - Socket/WCF/Rometing/Web Services

当今社会是一个信息化社会, 网络越来越多地影响人民的生活, 带来巨大的便利性; 而网络通信在软件开发中占据相当大的比重. Socket/WCF/Rometing/Web Services 各种技术纷至踏来, 应用广泛; 而...

长征2号
2017/08/11
0
0
通过文件流的方式上传文件

上传文件有四种方式 1:fileupload 2:ftp 3:网络共享 4:文件流写入 我们经常碰到这种困惑,大家都知道fileupload可以通过saveas方法存储文件,但是如果我们想把上传的文件写入到网络中其他...

邓小峰
2009/03/26
5.7K
0
CSharp HttpClient

这是一个 HTTP 客户端开发包 ,提供非常简单的 API,采用 C# 编写用来发送 HTTP 请求以及接收 HTTP 回应。 该库比 .NET 自带的 HttpWebRequest 要简单得多,类似 Java 版的 HttpClient。支持...

匿名
2011/12/16
5.3K
0
使用 HttpLib 来访问 Web 服务

介绍 C# 为开发者提供很多便利方法用来访问各种 Web 服务,但是编写异步的访问方式却是有点复杂。Windows 通讯框架是一个客户端/服务器通讯非常棒的工具。我精彩发现我自己必须写很多代码来使...

oschina
2013/01/02
4.3K
14

没有更多内容

加载失败,请刷新页面

加载更多

OpenStack 简介和几种安装方式总结

OpenStack :是一个由NASA和Rackspace合作研发并发起的,以Apache许可证授权的自由软件和开放源代码项目。项目目标是提供实施简单、可大规模扩展、丰富、标准统一的云计算管理平台。OpenSta...

小海bug
昨天
5
0
DDD(五)

1、引言 之前学习了解了DDD中实体这一概念,那么接下来需要了解的就是值对象、唯一标识。值对象,值就是数字1、2、3,字符串“1”,“2”,“3”,值时对象的特征,对象是一个事物的具体描述...

MrYuZixian
昨天
6
0
数据库中间件MyCat

什么是MyCat? 查看官网的介绍是这样说的 一个彻底开源的,面向企业应用开发的大数据库集群 支持事务、ACID、可以替代MySQL的加强版数据库 一个可以视为MySQL集群的企业级数据库,用来替代昂贵...

沉浮_
昨天
6
0
解决Mac下VSCode打开zsh乱码

1.乱码问题 iTerm2终端使用Zsh,并且配置Zsh主题,该主题主题需要安装字体来支持箭头效果,在iTerm2中设置这个字体,但是VSCode里这个箭头还是显示乱码。 iTerm2展示如下: VSCode展示如下: 2...

HelloDeveloper
昨天
7
0
常用物流快递单号查询接口种类及对接方法

目前快递查询接口有两种方式可以对接,一是和顺丰、圆通、中通、天天、韵达、德邦这些快递公司一一对接接口,二是和快递鸟这样第三方集成接口一次性对接多家常用快递。第一种耗费时间长,但是...

程序的小猿
昨天
9
0

没有更多内容

加载失败,请刷新页面

加载更多

返回顶部
顶部