Out of Memory Opening Very Large .xml file

Oxygen general issues.
jtwhaley
Posts: 2
Joined: Thu Mar 09, 2006 4:46 pm

Out of Memory Opening Very Large .xml file

Post by jtwhaley »

Hi everyone,

I'm attempting to open up a 1GB .xml that was extracted from another system. In doing so, I'm getting a OutOfMemoryError. I edited oxygen.bat and changed the line invoking the jva to read:


Code: Select all

java -Xmx2GB  -cp %CP% ro.sync.exml.Oxygen %1 %2 %3 %4 %5 %6 %7 
previous values of -Xmx tried were 256M and 512M.

My system's actual memory is only 512. Is there some other parameter I can alter to allow this monstrous xml file to be opened?

Thanks,
Jason[/code]
george
Site Admin
Posts: 2095
Joined: Thu Jan 09, 2003 2:58 pm

Post by george »

Hi Jason,

That is a large file indeed and clearly oXygen cannot handle a 1GB file on that machine. You should give oXygen maximum 256 MB of memory (more that this may cause the performace to decrease as your system will start using the disk for memory) and process documents up to 20 MB.

Best Regards,
George
george
Site Admin
Posts: 2095
Joined: Thu Jan 09, 2003 2:58 pm

Post by george »

And as a suggestion, it is possible to split that file in smaller fragments using either entities or XInclude. You can do that for instance if you write a program at SAX level and serialize the document in multiple files.

Best Regards,
George
jtwhaley
Posts: 2
Joined: Thu Mar 09, 2006 4:46 pm

Post by jtwhaley »

Thanks George,

It could be possible to split those xml files, but we will be working with them frequently and it would be a bit of a hassle do that each time.

Do you have any other suggestions?
george
Site Admin
Posts: 2095
Joined: Thu Jan 09, 2003 2:58 pm

Post by george »

In the last release of oXygen, 7.1 we added support for eXist and Berkeley DB XML databases. You may consider dropping your large file into such a database and then use XQuery to obtain data from your file.

Best Regards,
George
Post Reply