Stopping an XQuery Transform

[William]

Hi, I'm running oXygen XML Developer 14.0, build 2012062215 on Debian x64, while running an XQuery I noticed a mistake in my XPath so I decided to stop the transformation about 20 minutes in.

Some 7 minutes later it (oXygen) is still telling me "Transformation - is stopping ...". Now after (1994.6s) it tells me the transformation was successful! Do I detect a bug here? Surely this isn't correct as it's not what I would expect to happen.

--
William

[Costin]

Hello,

Could you please provide more details about the transformation you were trying to perform ? Also, what transformation engine are you using? Are you trying to perform the XQuery on local XML files or on remote files (over a database)? In case they are local, if you could share some sample files with us (XML and XQuery) for which this behavior occurs, should help us investigate the cause. If the files are not very large, please send them on our email - support(AT)oxygenxml(DOT)com

Feel free to address any further questions you might have.

Best regards,
Costin

[William]

Hello,

Could you please provide more details about the transformation you were trying to perform ?

The transformation is used to concatenate XML files into a single output while removing duplicates.

Also, what transformation engine are you using?

Saxon EE-9.4.0.3

Are you trying to perform the XQuery on local XML files or on remote files (over a database)?

I'm applying this to 3 files (4.3, 4,4 and 3.9)MB which generates an output of 25.8MB in 33 Minutes.

In case they are local, if you could share some sample files with us (XML and XQuery) for which this behavior occurs, should help us investigate the cause. If the files are not very large, please send them on our email - support(AT)oxygenxml(DOT)com

Feel free to address any further questions you might have.

Best regards,
Costin

The 3 files when tarballed take up 638KB and the XQuery file is 1.5KB if you would still like me to send these after reviewing the above then please let me know.

--
William

[Costin]

William,

Thank you for the details.
Sure, it would be great if you could email us the files for analysis.

Best,
Costin

[William]

William,

Thank you for the details.
Sure, it would be great if you could email us the files for analysis.

Best,
Costin

Sent.

[sorin_ristache]

Hello,

Thank you for the files.

Some 7 minutes later it (oXygen) is still telling me "Transformation - is stopping ...". Now after (1994.6s) it tells me the transformation was successful! Do I detect a bug here? Surely this isn't correct as it's not what I would expect to happen.

Your XQuery transformation needs such a long time because an in-memory tree needs to be built from the XML input and the total collective size of all the input XML files is 14 MB. The problem with an XML input this large is that building its tree in memory is very time consuming. This is why the streaming mode was invented for the transformation of XML files, which ensures that only a small part of the XML input is in memory at any time thus decreasing dramatically the memory and time requirements of the transformation. Unfortunately the streaming mode is available only for XSLT transformations, not XQuery ones.

I suggest rewriting the transformation for streaming mode, in XSLT instead of XQuery.


Regards,
Sorin

[William]

Hello,

Thank you for the files.

Some 7 minutes later it (oXygen) is still telling me "Transformation - is stopping ...". Now after (1994.6s) it tells me the transformation was successful! Do I detect a bug here? Surely this isn't correct as it's not what I would expect to happen.

Your XQuery transformation needs such a long time because an in-memory tree needs to be built from the XML input and the total collective size of all the input XML files is 14 MB. The problem with an XML input this large is that building its tree in memory is very time consuming. This is why the streaming mode was invented for the transformation of XML files, which ensures that only a small part of the XML input is in memory at any time thus decreasing dramatically the memory and time requirements of the transformation. Unfortunately the streaming mode is available only for XSLT transformations, not XQuery ones.

I suggest rewriting the transformation for streaming mode, in XSLT instead of XQuery.


Regards,
Sorin

Thanks for that Sorin, but it still doesn't explain why oXygen doesn't actually terminate the process! I understand from your reply why the process takes so long, fair enough, but that was never the issue. The issue was the fact that oXygen, as far as I am concerned, ignored my request to terminate the process; not how long the process took or would take.

When I clicked the button to terminate it the status bar update implied that oXygen was going to terminate the process. But as oXygen continued to the end it never did, thus defeating the purpose of actually terminating it and telling me there was a problem here.

So what do I get from this. Well this tells me that the parser engine is either not using a callback to deal with these requests, so the whole terminate option is pointless. Or, if there is a callback the callback isn't calling often enough, or the request isn't being propagated, or even something else.

--
William

[sorin_ristache]

Hi,

So what do I get from this. Well this tells me that the parser engine is either not using a callback to deal with these requests, so the whole terminate option is pointless. Or, if there is a callback the callback isn't calling often enough, or the request isn't being propagated, or even something else.

I am sorry, I overlooked your initial problem. The cause of the delayed stop is that the XQuery processor does not call the tracing callback often enough. Oxygen tries to stop the transformation at the first opportunity offered by the processor (that is when the processor will call the callback) after the moment of pressing the Stop button.


Regards,
Sorin

[William]

Hi,

I am sorry, I overlooked your initial problem. The cause of the delayed stop is that the XQuery processor does not call the tracing callback often enough. Oxygen tries to stop the transformation at the first opportunity offered by the processor (that is when the processor will call the callback) after the moment of pressing the Stop button.

Regards,
Sorin

That's OK and thanks for the reply. But the processor needs to be calling back a lot more often than it does. If it hasn't made a callback within 7 minutes then it's pointless even having it. But depending the the User experience you want to offer I would expect this to be done at least every second to make it as responsive as possible.

I appreciate that frequent callbacks have a processing overhead but from my own work not that much that the user would really notice.

Will this get updated in a future update / release?

--
William

[sorin_ristache]

Hi,

On closer inspection it seems you've just run into a bug in setting the callback listener. We will have to fix it and I will let you know by updating this discussion. If you need a maintenance build of Oxygen that includes a fix please let me know.


Best regards,
Sorin

[adrian]

Hello,

I just wanted to let you know that this problem has been resolved in the latest maintenance build of Oxygen 14.0, 2012082911 (released on August 30th):

XQuery: An XQuery transformation with the Saxon 9 processor could not be stopped in mid execution.

You can download it from our web site:
http://www.oxygenxml.com/download.html

The list of bug-fixes can be found here:
http://www.oxygenxml.com/build_history.html#2012082911

You can follow the release/build RSS feed here:
http://www.oxygenxml.com/rssBuildID.xml

Let us know if you encounter further problems with this new build.

Regards,
Adrian