Home | Contact Us | FAQ | Search & Site Map | Link to Us
Sign In | Join | Other 45 Sites in Network
HomeAnnouncementsFree MagazinesWhite PapersSubmit Content
Discussion GroupsASP.NETWindows FormsLanguages.NET FrameworkVisual Studio.NET
Articles.NET FrameworkASP.NETToolsWindows Forms
.NET DirectoryOpen Source ProjectsUser GroupsWeb Resources
Related Topics
Visual Basic 6SQL ServerMS AccessOther DB ProductsMS Server ProductsMore Topics ...

.NET Forum / Languages / C# / January 2008

Tip: Looking for answers? Try searching our database.

Huge xml files

Thread view: 
Enable EMail Alerts  Start New Thread
Thread rating: 
rickbear - 28 Jan 2008 15:04 GMT
Hi Group,

I have an xml file wich contains a rootelement and a subelement with a
filename and a subelement with bindata.
In each xml file there is only one entry for each element. Like this:

<root>
  <filename>somefile.exe</filename>
  <bindata>bindata in base64 here</bindata>
</root>

I have to parse the content in the bindata element to a Websphere MQ
queue, so I read the xml file into memory with xmlserializer and
filestream and put it in the queue. This works pretty fine.

On the other side of the queue I get the queuedata and collects the
data as an object. This works very well also.

But now and then there are xml files with huge amount of bindata. When
they exceed about 350mb I get an out of memory error. So I don't know
how to handle this. My idea is to somehow break the bindata into
smaller peaces. But I do not know how to partially load the data of
one element (bindata). Is that possible and how?
With this problem solved - is there some way to tell the message queue
that this is only a portion of an element?

- rick -
Ignacio Machin ( .NET/ C# MVP ) - 28 Jan 2008 15:49 GMT
Hi,

I think this is more a question for a Websphere MQ group, it depend of how
they can manage huge amount of data.
You can always read part of the file (maybe you will have to treat it as a
text file instead of a XML).

Signature

Ignacio Machin
http://www.laceupsolutions.com
Mobile & warehouse Solutions.

> Hi Group,
>
[quoted text clipped - 23 lines]
>
> - rick -
Jeroen Mostert - 28 Jan 2008 16:10 GMT
> Hi Group,
>
[quoted text clipped - 6 lines]
>    <bindata>bindata in base64 here</bindata>
> </root>

A spectacular misuse of XML for sure. This certainly doesn't beat putting
the actual files in a directory. Anyhoo...

> I have to parse the content in the bindata element to a Websphere MQ
> queue, so I read the xml file into memory with xmlserializer and
> filestream and put it in the queue. This works pretty fine.

Use XmlReader.ReadContentAsBase64() to read the content in chunks instead.
It was designed for this.

Signature

J.


Free Magazines

Get these publications absolutely FREE for up to 12 months. There are no hidden fees and no obligation. Simply choose a title, complete the application form and submit it. Read more ...

Oracle MagazineNetwork ComputingComputer WorldBio-IT WorldeWeekInformation WeekInfosecurity
 
Sign In
Join
My Latest Posts
My Monitored Threads
My Blog
My Photo Gallery
My Profile
My Homepage

Start New Thread
Enable EMail Alerts
Rate this Thread



©2008 Advenet LLC   Privacy Policy - Terms of Use
This website includes both content owned or controlled by Advenet as well as content owned or controlled by third parties.