Home | Contact Us | FAQ | Search & Site Map | Link to Us
Sign In | Join | Other 45 Sites in Network
HomeAnnouncementsFree MagazinesWhite PapersSubmit Content
Discussion GroupsASP.NETWindows FormsLanguages.NET FrameworkVisual Studio.NET
Articles.NET FrameworkASP.NETToolsWindows Forms
.NET DirectoryOpen Source ProjectsUser GroupsWeb Resources
Related Topics
Visual Basic 6SQL ServerMS AccessOther DB ProductsMS Server ProductsMore Topics ...

.NET Forum / .NET Framework / Performance / November 2006

Tip: Looking for answers? Try searching our database.

HTTP file performance

Thread view: 
Enable EMail Alerts  Start New Thread
Thread rating: 
smarty - 03 Nov 2006 09:51 GMT
Hi,

I am trying to take some latency measurement of HTTP download times for a
number of zip files varying in size from 100K to 500M.  I wrote some code
shown below but it produces out of memory errors when the files get up to
about 200M.  I am not sure if the code I have produced is what I want because
I used a streamreader that requires an encoding type to be specified, as its
a binary file type I don't think I should need to worry about encoding only
bits and bytes.  Can you make any suggestions on what would be the best way
acheive this please.

'setup local file to write to
Dim fs As New FileStream(LocalFilePath, FileMode.CreateNew,
FileAccess.Write, FileShare.None)

' Creates an HttpWebRequest for the specified URL.
Dim myHttpWebRequest As HttpWebRequest = CType(WebRequest.Create
(WebPageURL), HttpWebRequest)

' Sends the request and waits for a response.            
Dim myHttpWebResponse As HttpWebResponse = CType
(myHttpWebRequest.GetResponse(), HttpWebResponse)

' Calls the method GetResponseStream to return the stream associated with
the response.
Dim receiveStream As Stream = myHttpWebResponse.GetResponseStream()

' Pipes the response stream to a higher level stream reader with the
required encoding format.
Dim readStream As New StreamReader(receiveStream, encode)

pfileWriter.Write(readStream.Read())

Thanks
Joerg Jooss - 03 Nov 2006 21:27 GMT
Thus wrote smarty,

> Hi,
>
[quoted text clipped - 5 lines]
> type to be specified, as its a binary file type I don't think I should
> need to worry about encoding only bits and bytes.  
[...]

You *must not* use a StreamReader at all. It will only destroy your binary
payload, test or not ;-)

Worse, all of the broken binary content will end up as a string object, which
becomes garbage immediately.

You should instead consume the response stream using a small read buffer
(4-16 kB), and simply drop all bytes read.

// request is your HttpWebRequest
using(HttpWebResponse response = (HttpWebResponse) request.GetResponse())
using(Stream responseStream = response.GetResponseStream()) {
   byte[] buffer = new byte[0x1000];
   int bytes;
   while((bytes = responseStream.Read(buffer, 0, buffer.Length)) > 0) {
     // Do nothing ;-)
   }
}

Cheers,
Signature

Joerg Jooss
news-reply@joergjooss.de

Laura T - 05 Nov 2006 17:57 GMT
Any reason to not use WebClient.DownloadData/DownloadDataAsync,
or even better, WebClient.DownloadFile/DownloadFileAsync?

You'd get much better performance using those. And no memory problems.

> Hi,
>
[quoted text clipped - 34 lines]
>
> Thanks
Joerg Jooss - 05 Nov 2006 18:16 GMT
Thus wrote Laura,

> Any reason to not use WebClient.DownloadData/DownloadDataAsync, or
> even better, WebClient.DownloadFile/DownloadFileAsync?
>
> You'd get much better performance using those. And no memory problems.

Better performance where? It won't improve the HTTP throughput.

And why no memory problems? That has nothing to do with async I/O.

Signature

Joerg Jooss
news-reply@joergjooss.de

smarty - 05 Nov 2006 22:15 GMT
Hi,

Thanks to you both for your response.  I did use the WebClient class in the
end because I assumed this would have the most most efficient buffering
worked out automatically rather guessing what buffer size would be efficient.
Thanks for showing me how to setup the buffer code though I will find this
useful in the future and may still use it yet.

> Thus wrote Laura,
>
[quoted text clipped - 6 lines]
>
> And why no memory problems? That has nothing to do with async I/O.

Free Magazines

Get these publications absolutely FREE for up to 12 months. There are no hidden fees and no obligation. Simply choose a title, complete the application form and submit it. Read more ...

Oracle MagazineNetwork ComputingComputer WorldBio-IT WorldeWeekInformation WeekInfosecurity
 
Sign In
Join
My Latest Posts
My Monitored Threads
My Blog
My Photo Gallery
My Profile
My Homepage

Start New Thread
Enable EMail Alerts
Rate this Thread



©2008 Advenet LLC   Privacy Policy - Terms of Use
This website includes both content owned or controlled by Advenet as well as content owned or controlled by third parties.