Page 1 of 1

Intermittent Lock after Transform to PDF

Posted: Thu Nov 01, 2018 6:33 pm
by cud
I have set up transformation scenario that uses a template, plus CSS customizations, plus XSL customizations -- all to generate PDFs. It works wonderfully.

EXCEPT

Sometimes, and definitely not every time (maybe 30% to 50% of the time, depending on the day and maybe the files I'm transforming), oXygen freezes up and is unresponsive. If it happened every time I would assume it's my problem. But because it's intermittent, I have to think there's some edge case in what I'm doing that CAN trigger this behavior.

Mac OS 10.12.6
<oXygen/> XML Editor 20.1, build 2018080903

Is there a known issue I should be aware of? Has anybody else reported/seen this?

Thanks cud

Re: Intermittent Lock after Transform to PDF

Posted: Fri Nov 02, 2018 11:38 am
by Costin
Hello,

This could depend on the number and size of the files that are open in oXygen at the moment when you run the transformation scenario.
Oxygen could run out of memory in case you are working with large files, or with a large number of files.

You should try both increasing the available memory for oXygen and the allocated memory for your transformation.

For details on increasing the allocated memory, see the Setting Parameters for the Application Launchers section in the user-guide - look for "Increasing the amount of memory that Oxygen XML Editor uses" section corresponding to your operating system to increase the memory available to oXygen and in the Advanced Tab (DITA OT Transformations) - look for the "JVM Arguments parameter", to increase the memory for the DITA transformation scenarios.

Regards,
Costin

Re: Intermittent Lock after Transform to PDF

Posted: Fri Nov 02, 2018 1:31 pm
by cud
Thanks Costin. I'll look into that. I must say that I have experienced this when only the map file is open. Also, I have experienced this when only the map file is open, and it only includes 3 topic refs... And none of those topics include conrefs.

But it could well be that I need to give oXygen more heap. Could it also be that my JVM needs more heap in general? Does overall load on Java have an impact? (I run chronic java processes pretty regularly.)

Thanks cud

Re: Intermittent Lock after Transform to PDF

Posted: Fri Nov 02, 2018 7:00 pm
by cud
Ok, I added the following to my Info.plist file (note the first string in the array). I'm guessing this is what the array should look like after adding the string. Then I restarted. Then I ran the transform and it froze up on the first attempt. I wonder if there's a way to get some sort of diagnostics? Can I see if there's a memory problem going on? How and where?


<key>VMOptionArray</key>
<array>
<string>-Xmx1500m</string>
<string>-Dfile.encoding=UTF-8</string>
<string>-Dcom.oxygenxml.tree.readonly=false</string>
<string>-XX:-OmitStackTraceInFastThrow</string>
<string>-XX:SoftRefLRUPolicyMSPerMB=10</string>
<string>-Dsun.java2d.noddraw=true</string>
<string>-Dsun.awt.nopixfmt=true</string>
<string>-Dsun.java2d.dpiaware=true</string>
<string>-Djava.net.preferIPv4Stack=true</string>
<string>-Dsun.awt.keepWorkingSetOnMinimize=true</string>
<string>-Dinstall4j.launcherId=19</string>
<string>-Dinstall4j.swt=false</string>
<string>-Dexe4j.moduleName=$APP_PACKAGE</string>
</array>

Re: Intermittent Lock after Transform to PDF

Posted: Mon Nov 05, 2018 2:52 pm
by Costin
Hi cud,

Could you please also try increasing the memory allocated to the transformation?
Like I suggested in my previous post:
in the Advanced Tab (DITA OT Transformations) - look for the "JVM Arguments parameter"
You should try progressively increase the -Xmx value. For example, you could try with -Xmx512m, then -Xmx1024m a.s.o

Does this make any difference?

Regards,
Costin

Re: Intermittent Lock after Transform to PDF

Posted: Wed Nov 07, 2018 2:40 pm
by cud
So far so good. Because this was intermittent, I want to try normal use for a while and see what happens. But upping the heap to 1024 seems to have worked.

I wonder what makes my transforms use unusual amounts of memory? The actual files to transform are not huge. I do have one conref in some of them, and it pulls in about a page of UL items. I also run a custom XSLT against this extendion -- com.oxygenxml.pdf.css.xsl.merged2html5. Does branching into that somehow spike mem consumption?

Re: Intermittent Lock after Transform to PDF

Posted: Fri Nov 16, 2018 12:09 pm
by Dan
Maybe you take a look at the messages from the transformation console (Options/Preferences/DITA/Logging/Console). It might show up that the DITA-OT is processing more resources than expected.

Regards,
Dan

Re: Intermittent Lock after Transform to PDF

Posted: Fri Nov 16, 2018 9:14 pm
by cud
When a transform succeeds, I have no need to look at the console. When it fails, the product is frozen so I can't scroll. But I'll double check the next time it freezes.

Since I increased the mem for the transforms, I have had one episode of freezing... Once it freezes, then subsequent transforms also seem to freeze, even after restarting the app. Restarting the machine seems to help, though.

Final thing... I do get a "topic_file" is outside the scope of the input dita/map directory. For many reasons, I have my map files in a peer directory with the topic directories, not in a parent directory. It does transform, and it works with all of our other processes that we do. I can't imagine that drives my mem consumption by a factor of three, though.

Re: Intermittent Lock after Transform to PDF

Posted: Mon Nov 19, 2018 1:30 pm
by Costin
Hi cud,

As long as the peer folders are both local (it would be recommended to have all the resources in the same folder or in a child folder with the .ditamap file though), it should not have any influence on this.
We have logged an issue in our internal tracking system to perform some profiling on performance, so we could investigate if there are any memory leaks.

Best Regards,
Costin

Re: Intermittent Lock after Transform to PDF

Posted: Tue Nov 27, 2018 4:11 pm
by cud
I just got another lock-up. I don't think I can attach an image, but I took a picture of the screen.

At the bottom of the window it says,
[My_Transform] Transformation in progress. The progress bar is still there, and when you hover over the window you get the Mac "Not-Responding" pointer. But note that it does generate the PDF perfectly well... The PDF completes and opens in Acrobat.

For memory it says,
232 of 742 M

So I guess I'm not out of heap.

The console doesn't show anything. In fact, it says 0 problems. But when it successfully completes, my console does show warnings, related to the location of my map file compared to the topic files. So even though it creates the output successfully, it does not seem to finish posting errors. I wonder if that's any hint. (Well if that's what you mean by the console... The console tab is for "Transformation problems". So I guess it's not standard out...)

Re: Intermittent Lock after Transform to PDF

Posted: Wed Nov 28, 2018 10:35 am
by Radu
Hi Chris,

Interesting problem. If you edit the transformation scenario, in the "Output" tab you can uncheck the "Open in browser" checkbox to avoid having Oxygen open the PDF after the transformation finishes.
I'm curious if running the transformation without Oxygen having to open the PDF reader at the end will still produce that locking from time to time or not.

Regards,
Radu

Re: Intermittent Lock after Transform to PDF

Posted: Wed Nov 28, 2018 9:23 pm
by cud
Thanks Radu. I'll give it a try. I'll point out that my claim that it freezes before the console loads its warnings is not accurate. I ran some transforms that froze after the console loaded warnings. The case where it did NOT load warnings could have been a different map that doesn't elicit the warnings... I just can't be sure at this point.

Anyway, I'll try not displaying the PDF file to see how that works.

Re: Intermittent Lock after Transform to PDF

Posted: Mon Dec 17, 2018 11:29 pm
by cud
@Radu... I tried your suggestion of not opening the generated PDF file in a system editor (Reader). It seemed to work for a while. Sadly, that must have been coincidence, because I now get the same problem again. I cannot see any repeatable series of events or states that reliably causes this. I'm sorry to say that, believe me.

Re: Intermittent Lock after Transform to PDF

Posted: Tue Dec 18, 2018 11:16 am
by Radu
Hi Chris,

When Oxygen freezes, if you switch to another application and then back does Oxygen un-freeze? How unresponsive does Oxygen become when it freezes? Do the contextual menus works? Does the main menu work?

Regards,
Radu

Re: Intermittent Lock after Transform to PDF

Posted: Tue Dec 18, 2018 3:22 pm
by cud
Hi Radu. When I say unresponsive, I mean that the spinner is spinning, and you have to launch Activity Monitor (where it says it's unresponsive), and force quit. So no menus work. Going to other apps does not help. The program is completely unresponsive. I'm on the mac, BTW.

Re: Intermittent Lock after Transform to PDF

Posted: Tue Dec 18, 2018 6:24 pm
by adrian
Hello,

This sounds very similar to a known (although very rare) Java deadlock triggered by graphics switching. It is Mac specific, it can appear only on Macs with discrete GPU (AMD/nVidia dedicated graphics processor).
To verify this, please check for JavaApplicationStub*.hang log files in the Console app (Applications > Utilities), under ~/Library/Logs/DiagnosticReports. Please send them to our support email address, support@oxygenxml.com or use the Technical Support form from our web site.

I found an issue with the embedded Chemistry launcher that I believe could trigger graphics switching. The Chemistry Java process is not correctly run as a headless Java process from the DITA transformation scenario. Close to the end of the PDF transformation you may see a Java app icon pop in your dock bar for a short while and eventually disappear. You shouldn't see that, if Chemistry is correctly run as a headless process.
To work around this, edit the DITA scenario (duplicate if asked), go to the Advanced tab and in the JVM Arguments field add this parameter after the -Xmx (leave a space):

Code: Select all

-Djava.awt.headless=true
These JVM Arguments are also used by the embedded Chemistry launcher.
Run the modified DITA transformation scenario and you should no longer see the Java app icon appear in the dock bar. Hopefully this should also resolve the Oxygen hang (intermittent lock).

Oxygen only uses the integrated GPU, but Java processes that are not run in a certain manner always switch on the discrete GPU and thus can trigger this Java deadlock.
If the issue persists and graphics switching is indeed the trigger, one possible workaround is to disable graphics switching. Go to System Preferences > Energy Saver and clear the box for "Automatic graphics switching". This forces discrete graphics to stay enabled all the time, so it avoids the switch, but at a significant price, since discrete graphics is more power hungry. Only use this as a last resort.

Regards,
Adrian

Re: Intermittent Lock after Transform to PDF

Posted: Mon Dec 31, 2018 6:35 pm
by cud
Thanks Adrian...

Crossing my fingers and hoping the JVM arg fixes the issue (path of least resistance on my part).

Indeed, the Java icon used to show up, but it does not after setting the arg. And today I was having the same issue, but after setting this arg (but NOT restarting my machine -- somehow a restart makes the problem go away for a while) I no longer see the problem. So far so good, but since it's intermittent it will take time to believe that the arg really fixes the issue.

So, must ALL scenarios on Mac (with GPU) include this JVM arg? Or only scenarios to generate PDF? Or only scenarios that use Chemestry? Or...?

Finally, is there an obvious way to share scenario specs with other team members? Can I just push this modified scenario out to other people?

Re: Intermittent Lock after Transform to PDF

Posted: Mon Jan 07, 2019 12:45 pm
by adrian
Hi,

The "-Djava.awt.headless=true" Java argument is only necessary for the DITA PDF transformations that use Chemistry (PDF transformations based on CSS), if the Java icon popped up during the transformation.
Finally, is there an obvious way to share scenario specs with other team members? Can I just push this modified scenario out to other people?
There are several ways to deliver a scenario:
- Export/Import the scenario from the "Configure Transformation Scenario(s)" dialog (right click on scenario).
- If you have a custom DITA Map "Document Type Association" (framework) that you share among your team members, you can edit the "DITA Map PDF ... CSS" scenario within the Transformation tab of the "Document type" configuration dialog.
- If you have an Oxygen project file (.xpr) that is shared among your team members, you could just duplicate/edit the scenario as usual in the "Configure Transformation Scenario(s)" dialog and make sure the scenario has the Storage set to "Project Options".

Regards,
Adrian