Faster and/or more graceful handling of large XML files

Post by **jokester01au** » Wed Dec 20, 2006 9:28 am

Currently, when attempting to open an XML file that is more than a few megabytes, oxygen effectively hangs. This can result in loss of work and at the very least is rather annoying.

It would be good if oxygen could provide a progress bar and cancel button when attempting operations such as opening or validating large files so that the user can still interact with the application. The progress view of eclipse could be used to good effect here.

As an aside, if validate-as-you-type is taking a long time, it would be good for oxygen to offer to turn it off for the file you're currently working on.

Regards,
Joe.

Post by **jokester01au** » Wed Dec 20, 2006 9:30 am

Even worse, if you somehow end up in a situation where the workbench state contains an open xml file that is too large for oxygen to handle, eclipse will hang on loading. The only way I've found to get out of this situation is to remove the oxygen plugin, load eclipse and close the file that is too big.

Post by **george** » Wed Dec 20, 2006 5:38 pm

Hi,

We will look into that, however, note that you have progress on loading in the standalone version of oXygen. It works with the same license key.

Best Regards,
George

Post by **sorin_ristache** » Wed Dec 20, 2006 6:14 pm

Hello,

jokester01au wrote:It would be good if oxygen could provide a progress bar and cancel button when attempting operations such as opening or validating large files so that the user can still interact with the application.

Unfortunately the Eclipse framework does not allow plugins to customize the file open operation. The framework calls the plugin after the content of the file is loaded from disk in memory.

jokester01au wrote:if validate-as-you-type is taking a long time, it would be good for oxygen to offer to turn it off for the file you're currently working on.

Validate as you type can be disabled for all files from Window -> Preferences -> oXygen -> Editor -> Document Checking. We will consider adding this option also at file level.

jokester01au wrote:Even worse, if you somehow end up in a situation where the workbench state contains an open xml file that is too large for oxygen to handle, eclipse will hang on loading.

Eclipse reopens the files that were open in the previous Eclipse session and a plugin cannot change that.

Regards,
Sorin

Post by **jokester01au** » Thu Dec 21, 2006 2:06 am

Unfortunately the Eclipse framework does not allow plugins to customize the file open operation. The framework calls the plugin after the content of the file is loaded from disk in memory.

However, loading very large files in a basic text editor within eclipse is not a problem. It is only when loading the same file with the oxygen xml editor that eclipse hangs. This leads me to suspect that the time consuming operation must be the parsing which oxygen has to do to generate an outline and pretty printing.

Perhaps much of this could be saved in the workspace metadata?

Validate as you type can be disabled for all files from Window -> Preferences -> oXygen -> Editor -> Document Checking. We will consider adding this option also at file level.

Thanks, that's helpful. What I was suggesting is for oxygen to detect when validate as you type is taking too large a proportion of the processor's time, and proactively offering to switch it off.

Regards,
Joe.

Post by **sorin_ristache** » Thu Dec 21, 2006 10:21 am

Hello,

jokester01au wrote:However, loading very large files in a basic text editor within eclipse is not a problem. It is only when loading the same file with the oxygen xml editor that eclipse hangs.

We will try to reduce the loading time of large files.

Regards,
Sorin

Post by **kingargyle** » Thu Dec 21, 2006 7:57 pm

One thing I have noticed is that this can very greatly depending on platform and the particular JVM that is being used. My own experience with dealing with large XSDs under the Sun JVM on Windows under Eclipse, is that it can be very very slugish.

Using JRockit as the JVM with the -XXallocPrefetch and -XXallocRedoPrefetch options and Windows greatly speeds up the application. The biggest problem is Windows own memory manager. Even with about 1.2 gig of memory in the machine, my system would still end up using the swap file. Jrockit memory tweaking options are here:
http://edocs.bea.com/jrockit/jrdocs/ref ... ionXX.html

Under Ubunutu Linux, no swap file is used, and performance is acceptable on large files with the Sun JVM. It's even better with JRockit for Linux.

So, performance can greatly depend on a number of factors. Jrockit has optimized garbage collection abilities, and will adapt to how your application uses memory. With the added options mentioned above, performance and response can get to an acceptable level with Large XML files. With that said, there still could be optimizations done both in Oxygen, and within the Eclipse environment on memory usage.

Post by **kingargyle** » Thu Dec 21, 2006 7:59 pm

One other performance tip that will help in Eclipse. Is to turn off the Oxygen Code Folding feature. This will greatly increase the speed, especially in large files.

Post by **jokester01au** » Mon Nov 26, 2007 6:18 am

This is still a major issue with oxygen 9.

It can produce the following very irritating situation: An XML file is open in eclipse when it is exited. While eclipse is closed, another process changes that file and it becomes very large. Then eclipse will not start because oxygen spends forever trying to parse the very large file.

Eclipse has a progress view and system that is used for many other time consuming tasks. Surely it would make sense to utilise this for parsing oxygen files so that the user knows where it is up to and has the option to cancel.

Regards,
Joe.

Post by **sorin_ristache** » Mon Nov 26, 2007 11:24 am

Hello,

Yes, a progress indicator for the file open operation which allows canceling it should be presented. We will try to see how it can be implemented in a future version of the oXygen plugin.

Thank you for your suggestion,
Sorin