Webhelp publishing with DITA-OT 3.6

[jdupont]

I saw that DITA-OT 3.6 was released recently and that it provides process optimization (parallel + memory storage).
Our current webhelp publishing process takes a lot of time due to DITA-OT pre processes so I wanted to publish webhelp using DITA-OT 3.6
Unfortunatelly I get a lot of errors at the end of the process :

• XML utils not found from Ant project reference

• Store not found from Ant project reference

dita-ot-3.6-official\plugins\org.dita.base\build.xml:29: The following error occurred while executing this line:
dita-ot-3.6-official\plugins\com.oxygenxml.webhelp.responsive\build_dita.xml:591: The following error occurred while executing this line:
dita-ot-3.6-official\plugins\com.oxygenxml.webhelp.responsive\build_dita.xml:599: java.lang.NullPointerException

Any idea about to fix this issue?

[Radu]

Hi,

Our WebHelp Responsive user guide lists precisely what DITA OT version it works with, meaning that we thoroughly tested it with that specific DITA OT version.
We probably need to make changes on our side in the WebHelp plugins to add full support for DITA OT 3.6.
I'm actually working in updating our publishing plugins for DITA OT 3.6, so if nothing goes wrong probably Oxygen 23.1 (March this year) will come bundled with DITA OT 3.6.
If you would like to test a nightly build of Oxygen containing an updated DITA OT 3.6 publishing engine, please email us (support@oxygenxml.com) and I will contact you after we stabilize things on our side and run all our automated tests on the new integration.

A couple of questions:
- How many topics do you have and how much does the publishing take you right now?
- Are you publishing from Oxygen?
- Are your DITA topics DTD or RNG based?
- Are you publishing on Windows or on Mac/Linux?

For this latter question, from what I tested on my side for Windows, the memory storage parameter seemed to reduce the publishing by about half.
As for the parallel processing, for me at least it did not seem to influence the publishing time too much (maybe 5% off), and I tested it both on Windows and Mac.

Regards,
Radu

[jdupont]

Hello Radu,

Thank you for your feedback.
- How many topics do you have and how much does the publishing take you right now?
We have about 25000 topics and publishing takes about 4 hours.
- Are you publishing from Oxygen?
We publish from oXygen during dev and test phase. In production customer publishes using a server.
- Are your DITA topics DTD or RNG based?
Are DITA topics are based on a DTD. Would it make a difference to move to RNG?
- Are you publishing on Windows or on Mac/Linux?
We publish from Windows when we use oXygen.
When we publish from a server we are on a Linux env.

What you noticed with the "memory storage parameter" sounds impressive.
I will send you an email to make test on a nightly build of oXygen.

Thanks,
Jérôme

[Radu]

Hi Jérôme,

A couple of remarks/answers:

We have about 25000 topics and publishing takes about 4 hours.

Besides the "store-memory" and "parallel" parameters, in DITA OT 3.6 we also made small fixes intended to speed up publishing for very large projects:

https://github.com/dita-ot/dita-ot/issues/3568

so this will probably also help (at least a little bit) with your large project.

We publish from oXygen during dev and test phase. In production customer publishes using a server.

Does the server side publish with the same DITA OT publishing engine? Otherwise you will only get to benefit of the speed up on the client side.

Are DITA topics are based on a DTD. Would it make a difference to move to RNG?

Yes, the publishing would become slower The publishing engine needs to add more caches to speed up publishing of RNG-based topics and maps:
https://github.com/dita-ot/dita-ot/issues/3661

but Oxygen 23.1's bundled DITA OT will have these caches for RNG-based publishing added.

Rrgards,
Radu

[jdupont]

Hi Radu,

For your information we have also found one optimization on our side which can be interesting when we deal with a large data set.
We realized that most of the time was spent on the "move-meta-entries" step from DITA-OT.
Our assumption that was updating all the dita files with metadata takes a long time.
We tried to skip this step doing this:
<property name="preprocess.move-meta-entries.skip" value="true"/>

When we do this of course the process is much faster but the end result is not good (we don't have titles for instance).
We added one additional step on our content before we launch the publishing process.
We added a xsl stylesheet which moves all metadata to topics but in the ditamap files so instead of updating like 25000 ditafiles we only update about 100 ditamaps.
At the end the result looks almost ok... and the total publishing time is faster.

Regards,
Jérôme

[Radu]

Hi Jérôme,

Thanks for the update. I sent you via email the link with our latest DITA OT 3.6 + Oxygen WebHelp plugins.

Regards,
Radu