Faster and/or more graceful handling of large XML files

Are you missing a feature? Request it's implementation here.
jokester01au
Posts: 64
Joined: Tue May 17, 2005 9:16 am

Faster and/or more graceful handling of large XML files

Post by jokester01au » Wed Dec 20, 2006 9:28 am

Currently, when attempting to open an XML file that is more than a few megabytes, oxygen effectively hangs. This can result in loss of work and at the very least is rather annoying.

It would be good if oxygen could provide a progress bar and cancel button when attempting operations such as opening or validating large files so that the user can still interact with the application. The progress view of eclipse could be used to good effect here.

As an aside, if validate-as-you-type is taking a long time, it would be good for oxygen to offer to turn it off for the file you're currently working on.

Regards,
Joe.

jokester01au
Posts: 64
Joined: Tue May 17, 2005 9:16 am

Faster and/or more graceful handling of large XML files

Post by jokester01au » Wed Dec 20, 2006 9:30 am

Even worse, if you somehow end up in a situation where the workbench state contains an open xml file that is too large for oxygen to handle, eclipse will hang on loading. The only way I've found to get out of this situation is to remove the oxygen plugin, load eclipse and close the file that is too big.

george
Site Admin
Posts: 2101
Joined: Thu Jan 09, 2003 2:58 pm

Post by george » Wed Dec 20, 2006 5:38 pm

Hi,

We will look into that, however, note that you have progress on loading in the standalone version of oXygen. It works with the same license key.

Best Regards,
George
George Cristian Bina

sorin_ristache
Posts: 4144
Joined: Fri Mar 28, 2003 2:12 pm

Re: Faster and/or more graceful handling of large XML files

Post by sorin_ristache » Wed Dec 20, 2006 6:14 pm

Hello,
jokester01au wrote:It would be good if oxygen could provide a progress bar and cancel button when attempting operations such as opening or validating large files so that the user can still interact with the application.
Unfortunately the Eclipse framework does not allow plugins to customize the file open operation. The framework calls the plugin after the content of the file is loaded from disk in memory.
jokester01au wrote:if validate-as-you-type is taking a long time, it would be good for oxygen to offer to turn it off for the file you're currently working on.
Validate as you type can be disabled for all files from Window -> Preferences -> oXygen -> Editor -> Document Checking. We will consider adding this option also at file level.
jokester01au wrote:Even worse, if you somehow end up in a situation where the workbench state contains an open xml file that is too large for oxygen to handle, eclipse will hang on loading.
Eclipse reopens the files that were open in the previous Eclipse session and a plugin cannot change that.


Regards,
Sorin

jokester01au
Posts: 64
Joined: Tue May 17, 2005 9:16 am

Re: Faster and/or more graceful handling of large XML files

Post by jokester01au » Thu Dec 21, 2006 2:06 am

Unfortunately the Eclipse framework does not allow plugins to customize the file open operation. The framework calls the plugin after the content of the file is loaded from disk in memory.
However, loading very large files in a basic text editor within eclipse is not a problem. It is only when loading the same file with the oxygen xml editor that eclipse hangs. This leads me to suspect that the time consuming operation must be the parsing which oxygen has to do to generate an outline and pretty printing.

Perhaps much of this could be saved in the workspace metadata?
Validate as you type can be disabled for all files from Window -> Preferences -> oXygen -> Editor -> Document Checking. We will consider adding this option also at file level.
Thanks, that's helpful. What I was suggesting is for oxygen to detect when validate as you type is taking too large a proportion of the processor's time, and proactively offering to switch it off.

Regards,
Joe.

sorin_ristache
Posts: 4144
Joined: Fri Mar 28, 2003 2:12 pm

Re: Faster and/or more graceful handling of large XML files

Post by sorin_ristache » Thu Dec 21, 2006 10:21 am

Hello,
jokester01au wrote:However, loading very large files in a basic text editor within eclipse is not a problem. It is only when loading the same file with the oxygen xml editor that eclipse hangs.
We will try to reduce the loading time of large files.


Regards,
Sorin

kingargyle
Posts: 58
Joined: Fri Nov 11, 2005 6:35 am

Post by kingargyle » Thu Dec 21, 2006 7:57 pm

One thing I have noticed is that this can very greatly depending on platform and the particular JVM that is being used. My own experience with dealing with large XSDs under the Sun JVM on Windows under Eclipse, is that it can be very very slugish.

Using JRockit as the JVM with the -XXallocPrefetch and -XXallocRedoPrefetch options and Windows greatly speeds up the application. The biggest problem is Windows own memory manager. Even with about 1.2 gig of memory in the machine, my system would still end up using the swap file. Jrockit memory tweaking options are here:
http://edocs.bea.com/jrockit/jrdocs/ref ... ionXX.html

Under Ubunutu Linux, no swap file is used, and performance is acceptable on large files with the Sun JVM. It's even better with JRockit for Linux.

So, performance can greatly depend on a number of factors. Jrockit has optimized garbage collection abilities, and will adapt to how your application uses memory. With the added options mentioned above, performance and response can get to an acceptable level with Large XML files. With that said, there still could be optimizations done both in Oxygen, and within the Eclipse environment on memory usage.

kingargyle
Posts: 58
Joined: Fri Nov 11, 2005 6:35 am

Post by kingargyle » Thu Dec 21, 2006 7:59 pm

One other performance tip that will help in Eclipse. Is to turn off the Oxygen Code Folding feature. This will greatly increase the speed, especially in large files.

jokester01au
Posts: 64
Joined: Tue May 17, 2005 9:16 am

Post by jokester01au » Mon Nov 26, 2007 6:18 am

This is still a major issue with oxygen 9.

It can produce the following very irritating situation: An XML file is open in eclipse when it is exited. While eclipse is closed, another process changes that file and it becomes very large. Then eclipse will not start because oxygen spends forever trying to parse the very large file.

Eclipse has a progress view and system that is used for many other time consuming tasks. Surely it would make sense to utilise this for parsing oxygen files so that the user knows where it is up to and has the option to cancel.

Regards,
Joe.

sorin_ristache
Posts: 4144
Joined: Fri Mar 28, 2003 2:12 pm

Post by sorin_ristache » Mon Nov 26, 2007 11:24 am

Hello,

Yes, a progress indicator for the file open operation which allows canceling it should be presented. We will try to see how it can be implemented in a future version of the oXygen plugin.


Thank you for your suggestion,
Sorin

Post Reply