ReadToEnd

Thread view:

Enable EMail Alerts

Start New Thread

Thread rating:

AMercer - 13 Jul 2007 17:20 GMT

Below are two quoted pieces of the IO.StreamReader.ReadToEnd documentation
(remarks section), and my questions follow. These sections seem to
discourage using ReadToEnd and they leave me cold as to what to do instead.
FYI, my use of ReadToEnd is reading web pages from 10K to 1M bytes in size.

1. "ReadToEnd assumes that the stream knows when it has reached an end. For
interactive protocols, in which the server sends data only when you ask for
it and does not close the connection, ReadToEnd might block indefinitely and
should be avoided."

Come again? ReadToEnd should be avoided in what circumstances? If I
replace ReadToEnd with ReadBlock or Read, aren't I up against the same
problem?

2. "If the current method throws an OutOfMemoryException, the reader's
position in the underlying Stream is advanced by the number of characters the
method was able to read, but the characters already read into the internal
ReadLine buffer are discarded. Since the position of the reader in the stream
cannot be changed, the characters already read are unrecoverable, and can be
accessed only by reinitializing the StreamReader. If the initial position
within the stream is unknown or the stream does not support seeking, the
underlying Stream also needs to be reinitialized. To avoid such a situation
and produce robust code you should use the Read method and store the read
characters in a preallocated buffer"

I can get exceptions with Read, so the same worries apply. But this section
started with OutOfMemoryException, so maybe it is just talking about huge
streams that ought to be read in pieces. If so, why beat around the bushes
and why not just say so? If something else, then what?

Reply to this Message

Peter Duniho - 13 Jul 2007 18:08 GMT

> Below are two quoted pieces of the IO.StreamReader.ReadToEnd
> documentation
> (remarks section), and my questions follow. These sections seem to
> discourage using ReadToEnd

They do discourage using ReadToEnd() in certain situations, yes.

> and they leave me cold as to what to do instead.

You should instead use something that doesn't have those problems, if they
apply to you.

> FYI, my use of ReadToEnd is reading web pages from 10K to 1M bytes in
> size.

ReadToEnd() may or may not be suitable for your needs. It depends on how
you are using HTTP.

> 1. "ReadToEnd assumes that the stream knows when it has reached an
> end. For
[quoted text clipped - 7 lines]
> replace ReadToEnd with ReadBlock or Read, aren't I up against the same
> problem?

As the documentation states: in any circumstance where you are using an
interactive protocol in which the server sends data only when you ask for
it and does not close the connection. It's pretty simple.

As an example of this, consider using HTTP with persistent sessions, which
allows an HTTP request to be sent without closing the connection at the
end. The connection is left open for another request. Since ReadToEnd()
only completes when the end of the stream is reached, and since on a
network stream the "end of the stream" is defined as when the
stream-oriented connection is closed, using ReadToEnd() is not usually
suitable for use when you are using persistent sessions with HTTP (an
exception would be if you intend to send a series of predetermined
requests and then explicitly close the connection, resulting in the
responses for all requests to be returned in the ReadToEnd() method).

> 2. "If the current method throws an OutOfMemoryException, the reader's
> position in the underlying Stream is advanced by the number of
[quoted text clipped - 4 lines]
>
> I can get exceptions with Read, so the same worries apply.

No, they don't. As you already noted, this is specifically about the
OutOfMemoryException. In particular, ReadToEnd() may internally read a
little bit at a time, adding the data read to a growing buffer. If it
gets an exception trying to increase the buffer size, you will get an
exception rather than getting whatever data has been read so far. There
is no way at that point to access the data that's already been read. It's
been pulled from the stream already but has been lost.

Other exceptions may occur as well, and in fact they may also result in
you basically having to give up on that stream. IMHO, the main point of
this comment is to make clear that, unlike situations where you yourself
might get an OutOfMemoryException trying to allocate a buffer to use to
receive data, there is no way to recover from the exception in this case.
If you get the exception yourself, you can do things to try to free up
some memory and try again, without losing any data on the connection. But
if ReadToEnd() throws the exception, that's just as fatal as any of the
other exceptions that might be thrown.

> But this section
> started with OutOfMemoryException, so maybe it is just talking about huge
> streams that ought to be read in pieces.

IMHO, it has less to do with the length of the stream than it does with
how to deal with an out-of-memory exception. You don't need a long stream
in order to result in an out-of-memory exception, if you are already low
on memory.

> If so, why beat around the bushes
> and why not just say so? If something else, then what?

Again IMHO, the documentation is quite clear and they are saying exactly
what they intend it to say. You may find the information superfluous or
redundant, but the information _does_ provide a different view and insight
into what is happening than what is provided elsewhere in the
documentation. The issues are unique to ReadToEnd(), and they are things
that at least some people using ReadToEnd() would want to know.

Pete

Reply to this Message

Rate this thread:

ReadToEnd

Free Magazines