Large files crashes

[jakubm]

I am dealing with XML documents of size ~50MB and oxygen (14.2, windows) does not handle them very well.

1) When I try to open a file with the "optimize loading for large files" option, oxygen usually crashes:

Code: Select all

The application detected an error that is not automatically handled. It is possible that this error is not critical and the application may continue to work normally but our recommendation is to report the error and restart the application. If the application has become unstable and cannot be closed normally, you can use the Force Quit button from this dialog. Be aware that by doing this you will lose all unsaved changes in the opened documents.



Here are the error details:



[ Open dropped resource ]  -  java.lang.ArrayIndexOutOfBoundsException: 1048576

java.lang.ArrayIndexOutOfBoundsException: 1048576

	at ro.sync.document.f.charAt(Unknown Source)

	at ro.sync.document.s.f(Unknown Source)

	at ro.sync.document.s.c(Unknown Source)

	at ro.sync.document.o.jc(Unknown Source)

	at ro.sync.document.o.lb(Unknown Source)

	at ro.sync.exml.editor.b.d.dpf(Unknown Source)

	at ro.sync.exml.editor.b.d.ihf(Unknown Source)

	at ro.sync.exml.editor.n.zff(Unknown Source)

	at ro.sync.exml.editor.xmleditor.e.e.zff(Unknown Source)

	at ro.sync.exml.editor.kc.readDocument(Unknown Source)

	at ro.sync.exml.editor.validate.p.readDocument(Unknown Source)

	at ro.sync.exml.editor.xmleditor.rb.readDocument(Unknown Source)

	at ro.sync.exml.editor.qd.db(Unknown Source)

	at ro.sync.exml.editor.qd.u(Unknown Source)

	at ro.sync.exml.editor.kc.open(Unknown Source)

	at ro.sync.exml.editor.xmleditor.rb.open(Unknown Source)

	at ro.sync.exml.editor.kc.open(Unknown Source)

	at ro.sync.exml.editor.hd.mpm(Unknown Source)

	at ro.sync.exml.editor.hd.open(Unknown Source)

	at ro.sync.exml.e.u.b(Unknown Source)

	at ro.sync.exml.e.u.c(Unknown Source)

	at ro.sync.exml.editor.ob.fqm(Unknown Source)

	at ro.sync.exml.editor.hd.dqm(Unknown Source)

	at ro.sync.exml.editor.hd.brm(Unknown Source)

	at ro.sync.exml.editor.hd$5$1.ejd(Unknown Source)

	at ro.sync.ui.application.fb.run(Unknown Source)

2) The whole XML is without whitespace. When I try to use indent and format, oxygen usually runs out of assigned memory (1300MB). I don't know what magic oxygen does that it needs such a huge amount of mem just to indent a document.

I can upload the documents to an FTP if you want and tell me where.

[jakubm]

Also, running an XPath on a document without whitespace to find something is not very user friendly.

E.g. this expression
//*[text()='695149'

is found twice, i have the XPath locations, but I would like to jump to the occurences and I don't know how. Doubleclicking does not help (maybe because the occurence is on another page, but oxygen does not tell me on which one).

[adrian]

Hello,

On what platform are you running Oxygen (Windows, Linux, Mac)? If you're on Windows/Linux, what architecture/kit of Oxygen are you using (32/64 bit)?

1. If possible, please send us the files so we can analyze them and fix that error.
You can submit the files from the following page (please ZIP them before attaching):
http://www.oxygenxml.com/techSupport.html

2. This is the worst case scenario for XPath in Oxygen: very large document on a single line (no line breaks). This causes some limitations to be enforced even in "large file" mode, as documented here:
Oxygen User Manual - Editing Documents with Long Lines
The XPath location is very limited in this scenario (only the start tag is located). However, I see that this situation (no line breaks) is handled poorly when double clicking an item in the list, even in "large file" mode. I've logged an issue for this.
Note that for the "huge files" mode (that I believe you are using) there is currently no support available for jumping to the results of the XPath. This only works for the first page. The "huge files" mode is meant for files in excess of 300MB, but can also be useful for smaller documents that are very dense and have a very complex XML model that causes Oxygen to run out of memory. The limitations of this mode are documented here:
Oxygen User Manual - XML file sizes greater than 300 MB

If you can't format and indent the document after opening it (because it runs out of memory), there's a trick you can use to format it as it's being opened (this minimizes the memory used). Go to Options > Preferences, Editor / Format, and enable the option "Format and indent the document on open". Close and reopen the document and it will be formatted. Note that all opened documents will be formatted, so you will probably want to reset the option afterwards.

Regards,
Adrian

[jakubm]

Hi, I have send you a sample file.

It is a bit frustrating, I kind of ended up using Lister in Total Commander, which is very fast, but provides only simple text base find.

Oxygen really has troubles with large one-line documents and sometimes even when I add indentation with another tool, fails to load them anyway (please try the sample).

[jakubm]

Indent on open sometimes help (thanks for that hint).

[adrian]

Hi,

Thank you for the sample file. We have reproduced the problem and we will provide a bugfix in the next maintenance build of v14.2 (in a few weeks). We will notify you when it becomes available.

One way around the "large file" mode would be to increase the large file limit in the Oxygen options. However, this will also require a substantial amount of additional memory for Oxygen (which implies the use of the 64-bit edition).

The large file limit can be changed in: Options > Preferences, Editor / Open/Save, Optimize loading in the Text edit mode for files over (MB). You could set this to 70MB to fit your file.

To provide more memory for Oxygen, you should modify the -Xmx parameter in the oxygen14.2.vmoptions file from the Oxygen installation folder. Modify it to -Xmx2048m or -Xmx3096m (works only for 64-bit), but avoid using more than half of your physical RAM size.
Restart the application and verify the amount of memory that is actually available to Oxygen in Help -> About, JVM Memory Used/Total(last row).
In Windows Vista/7 you will first have to copy the oxygen14.2.vmoptions file to your Desktop(or another folder with write access), modify it there and then copy it back to the Oxygen installation folder replacing the original file.

Regards,
Adrian

[adrian]

Hello,

I just wanted to let you know that this problem (ArrayIndexOutOfBoundsException) has been resolved in the latest maintenance build of Oxygen 14.2, 2013030817 (released on March 12th):

Text mode: Fixed an ArrayIndexOutOfBoundsException that occurred in some cases when opening a large file.

You can download it from our web site:
http://www.oxygenxml.com/download.html

The list of bug-fixes can be found here:
http://www.oxygenxml.com/build_history.html#2013030817

You can follow the release/build RSS feed here:
http://www.oxygenxml.com/rssBuildID.xml

Let us know if you encounter further problems with this new build.

Regards,
Adrian