Home | Contact Us | FAQ | Search & Site Map | Link to Us
Sign In | Join | Other 45 Sites in Network
HomeAnnouncementsFree MagazinesWhite PapersSubmit Content
Discussion GroupsASP.NETWindows FormsLanguages.NET FrameworkVisual Studio.NET
Articles.NET FrameworkASP.NETToolsWindows Forms
.NET DirectoryOpen Source ProjectsUser GroupsWeb Resources
Related Topics
Visual Basic 6SQL ServerMS AccessOther DB ProductsMS Server ProductsMore Topics ...

.NET Forum / .NET Framework / New Users / February 2007

Tip: Looking for answers? Try searching our database.

Disabling chunking in HTTP 1.1

Thread view: 
Enable EMail Alerts  Start New Thread
Thread rating: 
joseph_gallagher@hotmail.com - 14 Feb 2007 11:43 GMT
Hi,

I'm not sure if this is the right place to post this but when
searching for the exception I got this group came up the most.

I'm making a HttpWebRequest to get data from a web page but am getting
the following error

System.Net.WebException: The server committed a protocol violation.
Section=ResponseBody Detail=Response chunk format is invalid
  at System.Net.HttpWebRequest.GetResponse()

I've looked in the groups and the recomended fix for this error is to
set

 <system.net>
   <settings>
     <httpWebRequest useUnsafeHeaderParsing="true" />
   </settings>
 </system.net>

in the App.config but this isnt working for me, playing around I find
that if I set the version in the HttpWebRequest to 1.0 like so

           HttpWebRequest hwr = (HttpWebRequest)
HttpWebRequest.Create(url.Text);
           hwr.ProtocolVersion = new System.Version(1,0);

then everything runs fine, running a packet sniffer I see this is
becuase I'm not recieving the data chunked, is there a header I can
set so I can carry on using 1.1 but not get chunked data as this is
obviously whats causing the problem and I dont really want to use http
1.0.

Failing that does anyone know what is causing this problem?
Joerg Jooss - 14 Feb 2007 19:05 GMT
Thus wrote joseph_gallagher@hotmail.com,

> Hi,
>
> I'm not sure if this is the right place to post this but when
> searching for the exception I got this group came up the most.

It's the right place :-)

> I'm making a HttpWebRequest to get data from a web page but am getting
> the following error
[quoted text clipped - 12 lines]
> in the App.config but this isnt working for me, playing around I find
> that if I set the version in the HttpWebRequest to 1.0 like so

That would be a severe bug on the server-side. Is this a public site? Or
is there a web proxy in between?

> HttpWebRequest hwr = (HttpWebRequest)
> HttpWebRequest.Create(url.Text);
[quoted text clipped - 4 lines]
> obviously whats causing the problem and I dont really want to use http
> 1.0.

Understandable. But it's the server's decision whether to use chunking or
not. A HTTP 1.1 client must support chunking, so there's no way to avoid
chunking without reverting to HTTP 1.0.

> Failing that does anyone know what is causing this problem?

Not without having a look at the actual traffic.

Cheers,
Signature

Joerg Jooss
news-reply@joergjooss.de

joseph_gallagher@hotmail.com - 15 Feb 2007 11:35 GMT
Thanks for the reply Joerg, it is a public server and I'm not going
through any proxies so it is strange, its also strange that it only
happens randomly but quiet frequently, around 50% of the time, some
sample code that calls the page in question is

       [STAThread]
       public static void Main() {
           string url = "http://site.sports.betfair.com/menu/
LoadMenuNodesAction.do?
sReturnPath=parent.frames['menu']&method=getMenuEvents&menuNodeId=11589568&strArrayName=allSkeletonArray&iParentID=11589568&layerName=allMarketsTreeContainer&strMenuPathArrayName=allPathArray&menuPathLayer=menuParents1&locale=en_GB";
           for(int i = 0 ; i < 10 ; i++) {
               HttpWebRequest hwr = (HttpWebRequest)
HttpWebRequest.Create(url);
               try {
                   using(WebResponse wr = hwr.GetResponse()) {
                       using(Stream s = wr.GetResponseStream()) {
                           using(StreamReader sr = new
StreamReader(s)) {
                               Console.WriteLine(sr.ReadToEnd());
                           }
                       }
                   }
               } catch(WebException we) {
                   Console.WriteLine(we.ToString());
               }
               Console.ReadLine();
           }
       }

Running a packet sniffer I can see that the correct text is been
returned but I'm not sure if it is been chunked correctly, its also
strange that the url works 100% in IE.

Thanks for any help, if you dont have time to look at it further dont
worry about it, usuing 1.0 is an acceptable alternative as its a call
that is only used occasionaly, I'm more just curious than anything
else.

Joe.
Alan J. McFarlane - 15 Feb 2007 14:06 GMT
> Thanks for the reply Joerg, it is a public server and I'm not going
> through any proxies so it is strange, its also strange that it only
> happens randomly but quiet frequently, around 50% of the time, some
> sample code that calls the page in question is

Seems to work ok always for me, at least I've not repro'd any error in
the number of times that I've run it...  Maybe someone else will find
different.  There's no chance there's a transparent proxy in your path,
see if there's an Via header in the response for instance.

If you've a sniffer trace of case where it fails can you upload that
somewhere and we can have a look to see if there's something obviously
wrong with the response.

The format of chunked is relatively simple { n n n CR LF {data of size
nnn} CR LF } for each chunk, with the end marked with a zero length
chunk, and the length 'nnn' is an ascii formatted number
so the first trace I have has this response:

HTTP/1.1 200 ....
...
Flags: ... BRefresh=onCRLF
CRLF
65CRLF
{65 bytes of content}CRLF
289CRLF
{289 bytes of content}CRLF
0CRLF
CRLF

Seems ok -- should do as it was a successful download.  (Not too sure
about the very last CRLF, but...)
Signature

Alan J. McFarlane
http://www.alanjmcf.me.uk/
Please follow-up in the newsgroup for the benefit of all.

Joerg Jooss - 16 Feb 2007 21:10 GMT
Thus wrote joseph_gallagher@hotmail.com,

> Thanks for the reply Joerg, it is a public server and I'm not going
> through any proxies so it is strange, its also strange that it only
> happens randomly but quiet frequently, around 50% of the time, some
> sample code that calls the page in question is

I can see some pretty strange artifacts here when enabling unsafe header
parsing -- it resembles a split HTTP response.

> [STAThread]
> public static void Main() {
[quoted text clipped - 25 lines]
> returned but I'm not sure if it is been chunked correctly, its also
> strange that the url works 100% in IE.

IE is much more lenient regarding HTTP issues than System.Net.

> Thanks for any help, if you dont have time to look at it further dont
> worry about it, usuing 1.0 is an acceptable alternative as its a call
> that is only used occasionaly, I'm more just curious than anything
> else.

Try to change your code as follows:

Instead of decoding the response using a StreamReader, simply dump the response
stream to a MemoryStream. After you've read the entire response, decode it
using System.Text.Encoding.UTF8.GetString(byte[]).

Cheers,
Signature

Joerg Jooss
news-reply@joergjooss.de


Rate this thread:







Free Magazines

Get these publications absolutely FREE for up to 12 months. There are no hidden fees and no obligation. Simply choose a title, complete the application form and submit it. Read more ...

Oracle MagazineNetwork ComputingComputer WorldBio-IT WorldeWeekInformation WeekInfosecurity
 
Sign In
Join
My Latest Posts
My Monitored Threads
My Blog
My Photo Gallery
My Profile
My Homepage

Start New Thread
Enable EMail Alerts
Rate this Thread



©2008 Advenet LLC   Privacy Policy - Terms of Use
This website includes both content owned or controlled by Advenet as well as content owned or controlled by third parties.