.NET Forum / .NET Framework / CLR / September 2007
starting streams at offsets
|
|
Thread rating:  |
Lee Crabtree - 24 Sep 2007 19:55 GMT Is there an existing means of starting a FileStream at a given offset into a file? That is, can the origin be set somewhere other than the beginning of the actual file?
As an example, let's suppose I'd like to disregard the first 32768 bytes in a file. I know I could create a normal file and just seek forward, but if I could set the origin of the stream 32768 bytes from the beginning of the file, I wouldn't have to know that offset anywhere other than the creation of the stream.
If it's not possible I'm not going to be doomed to failure, but it's something that would be handy.
Lee Crabtree
Doug Semler - 25 Sep 2007 02:02 GMT > Is there an existing means of starting a FileStream at a given offset into > a file? That is, can the origin be set somewhere other than the beginning [quoted text clipped - 8 lines] > If it's not possible I'm not going to be doomed to failure, but it's > something that would be handy. Why? Creating the stream *always* sets the offset to 0. You seek independently of the creation.
Stream s = (new whateverstream) s.Seek(offsetdesired, SeekOrigin.begin);
 Signature Doug Semler, MCPD a.a. #705, BAAWA. EAC Guardian of the Horn of the IPU (pbuhh). The answer is 42; DNRC o- Gur Hfrarg unf orpbzr fb shyy bs penc gurfr qnlf, abbar rira erpbtavmrf fvzcyr guvatf yvxr ebg13 nalzber. Fnq, vfa'g vg?
Jon Skeet [C# MVP] - 25 Sep 2007 08:51 GMT > > If it's not possible I'm not going to be doomed to failure, but it's > > something that would be handy. [quoted text clipped - 4 lines] > Stream s = (new whateverstream) > s.Seek(offsetdesired, SeekOrigin.begin); One point about the above code: I personally find the Position property clearer than the Seek method. It tends to be overlooked for historical reasons, but:
Stream s = (new whateverstream); s.Position = offsetDesired;
Just a bit simpler, IMO.
 Signature Jon Skeet - <skeet@pobox.com> http://www.pobox.com/~skeet Blog: http://www.msmvps.com/jon.skeet If replying to the group, please do not mail me too
Doug Semler - 26 Sep 2007 01:15 GMT > One point about the above code: I personally find the Position property > clearer than the Seek method. It tends to be overlooked for historical > reasons, but: > > Stream s = (new whateverstream); > s.Position = offsetDesired; Hehe, can you tell i come from the Unix world? I forgot about Position, because I am so used to random seeks from current position <g>
 Signature Doug Semler, MCPD a.a. #705, BAAWA. EAC Guardian of the Horn of the IPU (pbuhh). The answer is 42; DNRC o- Gur Hfrarg unf orpbzr fb shyy bs penc gurfr qnlf, abbar rira erpbtavmrf fvzcyr guvatf yvxr ebg13 nalzber. Fnq, vfa'g vg?
Jon Skeet [C# MVP] - 26 Sep 2007 19:29 GMT > > One point about the above code: I personally find the Position property > > clearer than the Seek method. It tends to be overlooked for historical [quoted text clipped - 5 lines] > Hehe, can you tell i come from the Unix world? I forgot about Position, > because I am so used to random seeks from current position <g> You're not alone - I see calls to Seek() far more often than calls to Position. The Position property is one of those "discovery" things - once you've seen it, you tend to use it, but a lot of people haven't seen it :)
 Signature Jon Skeet - <skeet@pobox.com> http://www.pobox.com/~skeet Blog: http://www.msmvps.com/jon.skeet If replying to the group, please do not mail me too
Doug Semler - 27 Sep 2007 00:40 GMT >> > One point about the above code: I personally find the Position property >> > clearer than the Seek method. It tends to be overlooked for historical [quoted text clipped - 10 lines] > once you've seen it, you tend to use it, but a lot of people haven't > seen it :) Not sure if it's applicable to this group but I was curious about it. Is there any performance hit by using the "Position" property versus "Seek(offset, SeekOffset.Current)?
Reason I ask: Example, the FileStream class Position property resets the seek from the beginning of the file regardless of the current position...just wondering if the reset back to the beginning of the file hurts performance any versus a seek from the current offset...
(BTW I know what you mean about "discovery" things....just found out about the ?? operator the other day ... thanks for mentioning it!!! lol)
 Signature Doug Semler, MCPD a.a. #705, BAAWA. EAC Guardian of the Horn of the IPU (pbuhh). The answer is 42; DNRC o- Gur Hfrarg unf orpbzr fb shyy bs penc gurfr qnlf, abbar rira erpbtavmrf fvzcyr guvatf yvxr ebg13 nalzber. Fnq, vfa'g vg?
Jon Skeet [C# MVP] - 27 Sep 2007 17:35 GMT > > You're not alone - I see calls to Seek() far more often than calls to > > Position. The Position property is one of those "discovery" things - [quoted text clipped - 4 lines] > there any performance hit by using the "Position" property versus > "Seek(offset, SeekOffset.Current)? I don't know - I would expect there to be any though.
> Reason I ask: Example, the FileStream class Position property resets the > seek from the beginning of the file regardless of the current > position...just wondering if the reset back to the beginning of the file > hurts performance any versus a seek from the current offset... It doesn't reset back to the beginning of the file - it resets *relative* to the beginning of the file, i.e. an "absolute" position. Assuming the OS maintains some sort of integer to say where the stream currently is, it could easily optimise an absolute call to a relative one if that gives a performance improvement in a particular filesystem.
I guess if you were doing a lot of this it and had performance issues would be worth benchmarking, but I wouldn't expect to see a lot of difference.
> (BTW I know what you mean about "discovery" things....just found out about > the ?? operator the other day ... thanks for mentioning it!!! lol)
:)
 Signature Jon Skeet - <skeet@pobox.com> http://www.pobox.com/~skeet Blog: http://www.msmvps.com/jon.skeet If replying to the group, please do not mail me too
Doug Semler - 28 Sep 2007 23:56 GMT >> > You're not alone - I see calls to Seek() far more often than calls to >> > Position. The Position property is one of those "discovery" things - [quoted text clipped - 21 lines] > would be worth benchmarking, but I wouldn't expect to see a lot of > difference. That's what I wasn't sure about. Some (older) OSs that I had worked with in the past (notably earlier versions of HP-UX and I think SunOS 4 IIRC) would basically invalidate the I/O cache for the file if you did a seek from begin over current because it made different assumptions based on Begin, Current, End...whereas a forward look/backward look off a current offset may or may not have have invalidated the cache (depending on the size of the seek, etc). But that may have been more of a hardware architecture layout than anything else)
in other words, if your current fp was located at say offset 32,127,181
seek(32,100,101, BEGIN) would often cause a cache flush whereas seek(-27,180, CURRENT) more often than not did not...
(but, like I said, Windows wasn't the OS I grew up on, and I/O is not the bottlenecks in image processing which is my current balliwick. :)
 Signature Doug Semler, MCPD a.a. #705, BAAWA. EAC Guardian of the Horn of the IPU (pbuhh). The answer is 42; DNRC o- Gur Hfrarg unf orpbzr fb shyy bs penc gurfr qnlf, abbar rira erpbtavmrf fvzcyr guvatf yvxr ebg13 nalzber. Fnq, vfa'g vg?
Lee Crabtree - 25 Sep 2007 16:06 GMT Here's a more concrete example of what I'm currently tasked with doing.
Multiple images have been put into one file, with a "master" header that gives the offsets into the file to the beginning of each image. Now, I COULD use a single stream and seek to the beginning of each file when I need to, but it seems to me that it would make MUCH more sense to use one stream for each image (everything is reading only, so I don't have to worry about writing) with each stream's origin being the offset into the file of the associated image.
If that's not possible, I'm basically forced to hold multiple offsets and jump a single stream around the file, and I can just see that being all kinds of error-prone.
Lee Crabtree
>> Is there an existing means of starting a FileStream at a given offset >> into a file? That is, can the origin be set somewhere other than the [quoted text clipped - 14 lines] > Stream s = (new whateverstream) > s.Seek(offsetdesired, SeekOrigin.begin); Günter Prossliner - 25 Sep 2007 16:48 GMT Hello Lee!
> I COULD use a single stream and seek to the beginning of each file > when I need to, but it seems to me that it would make MUCH more sense > to use one stream for each image (everything is reading only, so I > don't have to worry about writing) with each stream's origin being > the offset into the file of the associated image. Of cause it is possible to do so. You may implement a special System.IO.Stream which represents another stream starting from offset X.
e.g (untested).:
class OffsetStream : Stream {
Stream _s; long _o, _l;
public OffsetStream(Stream stream, long offset, long length){ _s = stream; _o = offset; _l = length;
if(!_s.CanSeek) throw new ArgumentException("not seekable..."); }
public override bool CanRead{ return _s.CanRead; }
// do so with "CanWrite", "CanSeek", ...
public override long Length { get{return _l;} }
public override void SetLength(long value){ // would affect other parts of the stream throw new NotSupportedException(); }
// seeking
public override long Seek(long offset, SeekOrgin orgin){ long p; switch(orgin){ case SeekOrgin.Begin: p = _o+offset; break; case SeekOrgin.Current: p = Position + offset; case SeekOrgin.End: p = _o + _l - offset; break; }
_s.Seek(SeekOrgin.Begin, p); return p - _o; }
public override long Position { get{ return _s.Position - _o; } set{Seek(value, SeekOrgin.Begin);} }
// implement the other abstract members }
This class may not be used to split up an Stream and to concurrenlty use the parts individually (even in single-threaded environments). Because all "OffsetStream" instances manipulate the Postion within the supplied Stream, they would affect each other.
You have to know what operations are happening with your "OffsetStream" to know if this is a problem or not. Image.FromStream() is a "Black-Box". It is not documented when and how the Stream is used. So I would NOT recommend to use the "OffsetStream" in this scenario.
You may implement the "OffsetStream" in a way that it loads all Data into memory when it gets constructed. This would actually increase the Memory-Footprint, but each Stream can operate individually.
e.g.
class OffsetStream : MemoryStream {
public OffsetStream(Stream stream, long offset, long length) : base(GetData(stream, offset, length), false) {}
static byte[] GetData(Stream stream, long offset, long length){
byte[] buffer = new byte[(int)length]; stream.Read(buffer, (int)offset, buffer.Length); return buffer;
// you should check the return-value of "stream.Read" to ensure that // the data is read completly.
// you can also use a "BinaryReader" (which checks the Return-Value by itself): BinaryReader reader = new BinaryReader(stream);
// allthough this lock does NOT garanty that this code is thread-safe in any case, // it garanties that it is threadsafe when you create multiply OffsetStream instances // based on the the same Stream-Object. lock(stream){ stream.Position = offset; return reader.ReadBytes((int)length); } }
}
GP
Free MagazinesGet these publications absolutely FREE for up to 12 months. There are no hidden fees and no obligation. Simply choose a title, complete the application form and submit it. Read more ...
|
|
|