oXygen DITA to WebHelp automated transformation

[vitalii_samoilenko]

Hi all,

I'm trying to automate DITA to WebHelp transformation using DITA-OT 1.8 toolkit bundled within oXygen XML Editor 18.0.
I already have oXygen XML WebHelp license purchased and applied there.

At this point I'm able to kick off the transformation via command line by invoking ANT build using Java.
It does some stuff, emits tons of log entries and working files.

At some point of execution process it calls dita.out.map.webhelp.toc target to customize table of contents.
There are several XSLT transformation invocations which are failing with the following error:

Code: Select all

...

[xslt] C:\Program Files (x86)\Oxygen XML Editor 18\frameworks\dita\DITA-OT\plugins\com.oxygenxml.webhelp\xsl\localization.xsl:53:104: Fatal Error! Cannot find a matching 1-argument function named {java:java.io.File}new(). Reflexive calls to Java methods require a Saxon-PE or Saxon-EE license, and none was found

[xslt] C:\Program Files (x86)\Oxygen XML Editor 18\frameworks\dita\DITA-OT\plugins\com.oxygenxml.webhelp\xsl\localization.xsl:61:88: Fatal Error! Cannot find a matching 1-argument function named {java:java.io.File}new(). Reflexive calls to Java methods require a Saxon-PE or Saxon-EE license, and none was found

...

From the description above I assume that I should buy a corresponding license.

Initially, since I'm new to oXygen, DITA-OT and Saxon, I made some digging inside the process itself and noted a target named check-license defined within WebHelp build files:

Code: Select all

    <target name="check-license" unless="${env.SAXON_EE_LICENSED_ENV}">

      <echo>Using directory ${webhelp.lib.dir} </echo>

        <available classname="ro.sync.licensemanager.WebhelpLicenseChecker" 

            property="WebhelpLicenseChecker.present">

            <classpath>

                <pathelement path="${webhelp.lib.dir}/license.jar"/>

                <pathelement path="${webhelp.lib.dir}/log4j.jar"/>

                <pathelement path="${user.dir}/lib/license.jar"/>

                <pathelement path="${user.dir}/lib/log4j.jar"/>

            </classpath>                	

        </available>

        <fail unless="WebhelpLicenseChecker.present" 

            message="The WebHelp Plugin does not contain all jar files. Please contact support@oxygenxml.com for more details."/>

        <java classname="ro.sync.licensemanager.WebhelpLicenseChecker" 

            failonerror="true"

            fork="true">

            <arg value="${webhelp.dir}"/>

            <classpath>

                <pathelement path="${webhelp.lib.dir}/license.jar"/>

                <pathelement path="${webhelp.lib.dir}/log4j.jar"/>

                <pathelement path="${user.dir}/lib/license.jar"/>

                <pathelement path="${user.dir}/lib/log4j.jar"/>

            </classpath>

        </java>

    </target>

As we may see it checks environment variable related to Saxon EE licensing and looks for oXygen XML WebHelp license if it's missing.

With all that said and discovered I become confused since it looks like I could have only Saxon EE license onboard without oXygen XML WebHelp one.

Could please somebody clarify what exact license(s) should I have to accomplish my task?

Thanks and Regards,
Vitalii.

[Radu]

Hi Vitalii,

Oxygen comes bundled with the latest Saxon EE commercial processor but according to our contract with the Saxon manufacturer the processor can only be used when running processes from inside Oxygen.
When running DITA OT 1.8 outside of Oxygen you can use an older version of Saxon 9 which was free and allowed calls to Java extension methods.
The DITA OT 1.8 bundled with Oxygen is missing those older Saxon 9 libraries but you can download separately a DITA OT 1.8 directly from the web site:

http://sourceforge.net/projects/dita-ot ... 20Release/

and then either copy the extra JAR libraries it has in its "lib" folder to the DITA OT 1.8 bundled with Oxygen or instead of using the DITA OT 1.8 bundled with Oxygen follow the procedures here:

https://www.oxygenxml.com/doc/versions/ ... ta-ot.html

Regards,
Radu

[vitalii_samoilenko]

Hi Radu,

I've downloaded DITA-OT1.8.5_full_easy_install.zip, copied Saxon libraries into oXygen lib folder and referenced saxon9.jar instead of bundled saxon9ee.jar.
After that build started to fail before the previous point, at target mappull with following error:

Code: Select all

mappull:

	...

	[mappull] Loading stylesheet C:\Program Files (x86)\Oxygen XML Editor 18\frameworks\dita\DITA-OT\xsl\preprocess\mappull.xsl

	[mappull] C:\Program Files (x86)\Oxygen XML Editor 18\frameworks\dita\DITA-OT\plugins\com.oxygenxml.webhelp\xsl\dita\addResourceID.xsl:21: Fatal Error! Unknown XSLT element: sequence

	[mappull] Failed to process null

	...

It seems like that this element belongs to XSLT 2.0 and higher and we lost ability to use it after applying free distributed Saxon.

Any help with this would be appreciated.

Thanks and Regards,
Vitalii.

[Radu]

Hi Vitalii,

So:

I've downloaded DITA-OT1.8.5_full_easy_install.zip, copied Saxon libraries into oXygen lib folder and referenced saxon9.jar instead of bundled saxon9ee.jar.

You should have copied the Saxon libraries from the downloaded DITA OT 1.8.5 to:
OXYGEN_INSTALL_DIR\frameworks\dita\DITA-OT\lib

You also mentioned that you changed a reference from saxon9ee.jar to saxon9.jar. Where did you do that?


Regards,
Radu

[vitalii_samoilenko]

For testing purposes I've tried to use Saxon libraries from different locations:

Code: Select all

OXYGEN_INSTALL_DIR\frameworks\dita\DITA-OT\lib\saxon (original one)

OXYGEN_INSTALL_DIR\frameworks\dita\DITA-OT\lib

OXYGEN_INSTALL_DIR\lib\saxon

OXYGEN_INSTALL_DIR\lib

However, at this point nothing helped to solve it.

Regarding saxon.jar reference, prevoiusly I told that I invoke ANT build using Java.
Here's the command line I'm using:

Code: Select all

set OXYGEN_PATH=C:\Program Files\Oxygen XML Editor 18

set CURDIR=C:\TestDitaIn

set OUTDIR=C:\TestDitaOut



"%OXYGEN_PATH%\jre/bin/java"^

 -Xmx384m^

 "-Dorg.apache.xerces.xni.parser.XMLParserConfiguration=ro.sync.xml.transformer.dita.remote.RelaxDefaultsDITARemoteMapsParserConfiguration"^

 -classpath "%OXYGEN_PATH%/tools/ant/lib/ant-launcher.jar"^

 "-Dant.home=%OXYGEN_PATH%/tools/ant"^

 org.apache.tools.ant.launch.Launcher^

 -lib "%OXYGEN_PATH%\lib\oxygen.jar"^

 -lib "%OXYGEN_PATH%\lib\oxygen-annotations.jar"^

 -lib "%OXYGEN_PATH%\lib\oxygen-content-completion-api.jar"^

 -lib "%OXYGEN_PATH%\lib\oxygen-css-pretty-printer.jar"^

 -lib "%OXYGEN_PATH%\lib\oxygen-css-validator.jar"^

 -lib "%OXYGEN_PATH%\lib\oxygen-emf.jar"^

 -lib "%OXYGEN_PATH%\lib\oxygen-text-search.jar"^

 -lib "%OXYGEN_PATH%\lib\oxygen-token-markers.jar"^

 -lib "%OXYGEN_PATH%\lib\oxygen-validation-api.jar"^

 -lib "%OXYGEN_PATH%\lib\oxygen-xquery-pretty-printer.jar"^

 -lib "%OXYGEN_PATH%\lib\resolver.jar"^

 -lib "%OXYGEN_PATH%\lib\org.eclipse.wst.xml.xpath2.processor_1.2.0.jar"^

 -lib "%OXYGEN_PATH%\lib\xml-apis.jar"^

 -lib "%OXYGEN_PATH%\lib\xercesImpl.jar"^

 -lib "%OXYGEN_PATH%\lib\commons-io-1.3.1.jar"^

 -lib "%OXYGEN_PATH%\lib\commons-logging-1.2.jar"^

 -lib "%OXYGEN_PATH%\lib\log4j.jar"^

 -lib "%OXYGEN_PATH%\lib\jing.jar"^

 -lib "%OXYGEN_PATH%\lib\saxon.jar"^

 -lib "%OXYGEN_PATH%\lib\saxon\saxon9.jar"^

 -lib "%OXYGEN_PATH%\lib\xalan.jar"^

 -lib "%OXYGEN_PATH%\lib\xmlgraphics-commons-2.1.jar"^

 -lib "%OXYGEN_PATH%\lib\fop.jar"^

 -lib "%OXYGEN_PATH%\lib\batik-all-1.8.jar"^

 -lib "%OXYGEN_PATH%\lib\js.jar"^

 -lib "%OXYGEN_PATH%\lib\poi-3.10-FINAL-20140208.jar"^

 -lib "%OXYGEN_PATH%\lib\nekohtml.jar"^

 -lib "%OXYGEN_PATH%\lib\xml-apis-ext.jar"^

 -lib "%OXYGEN_PATH%\lib\avalon-framework-api-4.3.1.jar"^

 -lib "%OXYGEN_PATH%\lib\avalon-framework-impl-4.3.1.jar"^

 -lib "%OXYGEN_PATH%\lib\jeuclid-core.jar"^

 -lib "%OXYGEN_PATH%\lib\jeuclid-fop.jar"^

 -lib "%OXYGEN_PATH%\lib\jai_tiff.jar"^

 -lib "%OXYGEN_PATH%\lib\jh.jar"^

 -lib "%OXYGEN_PATH%\lib\lucene-analyzers-common-4.0.0.jar"^

 -lib "%OXYGEN_PATH%\lib\lucene-core-4.0.0.jar"^

 -lib "%OXYGEN_PATH%\lib\lucene-queries-4.0.0.jar"^

 -lib "%OXYGEN_PATH%\lib\lucene-queryparser-4.0.0.jar"^

 -lib "%OXYGEN_PATH%\lib\lucene-suggest-4.0.0.jar"^

 -lib "%OXYGEN_PATH%\lib\commons-httpclient-3.1.jar"^

 -lib "%OXYGEN_PATH%\lib\httpclient-4.5.1.jar"^

 -lib "%OXYGEN_PATH%\lib\httpcore-4.4.3.jar"^

 -lib "%OXYGEN_PATH%\lib\httpclient-cache-4.5.1.jar"^

 -lib "%OXYGEN_PATH%\lib\fluent-hc-4.5.1.jar"^

 -lib "%OXYGEN_PATH%\lib\httpmime-4.5.1.jar"^

 -lib "%OXYGEN_PATH%\lib\truezip-6.jar"^

 -lib "%OXYGEN_PATH%\lib\jide.jar"^

 -lib "%OXYGEN_PATH%\lib\jsch-0.1.51.jar"^

 -lib "%OXYGEN_PATH%\frameworks\dita\DITA-OT"^

 -lib "%OXYGEN_PATH%\frameworks\dita\DITA-OT\lib"^

 -lib "%OXYGEN_PATH%\frameworks\dita\DITA-OT\lib\dost-patches.jar"^

 -lib "%OXYGEN_PATH%\frameworks\dita\DITA-OT\lib\dost.jar"^

 -lib "%OXYGEN_PATH%\frameworks\dita\DITA-OT\lib\commons-codec-1.4.jar"^

 -lib "%OXYGEN_PATH%\frameworks\dita\DITA-OT\plugins\com.oxygenxml.webhelp\lib\xhtml-indexer.jar"^

 -lib "%OXYGEN_PATH%\frameworks\dita\DITA-OT\plugins\com.oxygenxml.highlight\lib\xslthl-2.1.1.jar"^

 -f "%OXYGEN_PATH%\frameworks/dita/DITA-OT/build.xml"^

 "-Dtranstype=webhelp"^

 "-Dbasedir=%CURDIR%"^

 "-Doutput.dir=%OUTDIR%"^

 "-Dwebhelp.trial.license=no"^

 "-Ddita.temp.dir=%OUTDIR%/temp"^

 "-Dargs.hide.parent.link=no"^

 "-Ddita.dir=%OXYGEN_PATH%\frameworks/dita/DITA-OT"^

 "-Dargs.xhtml.classattr=yes"^

 "-Douter.control=quiet"^

 "-Dfix.external.refs.com.oxygenxml=true"^

 "-Dgenerate.copy.outer=2"^

 "-Dwebhelp.footer.include=no"^

 "-Dclean.output=yes"^

 "-Dargs.input=%CURDIR%\Test.ditamap"^

 "-Dwebhelp.copyright="^

 "-DbaseJVMArgLine=-Xmx384m"

It's similar to what oXygen XML editor dumps during transformation.

Previously it had following reference:

Code: Select all

 -lib "%OXYGEN_PATH%\lib\saxon9ee.jar"^

Which I assume is that latest Saxon processor You mentioned before.

So I changed it to be:

Code: Select all

 -lib "%OXYGEN_PATH%\lib\saxon\saxon9.jar"^

Thanks and Regards,
Vitalii.

[Radu]

Hi Vitalii,

Looking at the order of these 2 libraries in your classpath:

Code: Select all

 -lib "%OXYGEN_PATH%\lib\saxon.jar"^

 -lib "%OXYGEN_PATH%\lib\saxon\saxon9.jar"^

it seems that you are adding Saxon 6 before Saxon 9 and thus all transformations will use Saxon 6 which is not XSLT 2.0 compliant.
So you should either remove the reference saxon.jar from the classpath or add in the classpath first saxon9.jar and then saxon.jar.

Regards,
Radu

[vitalii_samoilenko]

Thank You, that worked to solve the issue with sequence element.

However, it's still failing.
Now I get the following error:

Code: Select all

...

mappull:

  ...

  [mappull] Loading stylesheet C:\Program Files\Oxygen XML Editor 18\frameworks\dita\DITA-OT\xsl\preprocess\mappull.xsl

  [mappull] Unknown file:26:91: Fatal Error! Could not find function: base-uri

  [mappull] : Fatal Error! Fatal error during transformation using C:\Program Files\Oxygen XML Editor 18\frameworks\dita\DITA-OT\xsl\preprocess\mappull.xsl: Could not find function: base-uri; Line#: 26; Column#: 91 Cause: Fatal error during transformation using C:\Program Files\Oxygen XML Editor 18\frameworks\dita\DITA-OT\xsl\preprocess\mappull.xsl: Could not find function: base-uri; Line#: 26; Column#: 91

  [mappull] Failed to process null

...

I think I may have another mess with libraries.
I've tried to reference all Saxon libraries that come with free DITA-OT (like saxon9-dom.jar, saxon9-s9api.jar and so on) but it didn't fix things.

Could You please help me to identify the problem?

Thanks and Regards,
Vitalii.

[Radu]

Hi Vitalii,

Please add in the classpath list the reference to saxon9.jar before the reference to the "-lib "oxygen.jar". The reason would be that the main oxygen JAR contains patches destined exclusively for the Saxon EE which comes with Oxygen. Or you can remove all references to the main "oxygen.jar".
You should also remove the custom parser configuration:

Code: Select all

 "-Dorg.apache.xerces.xni.parser.XMLParserConfiguration=ro.sync.xml.transformer.dita.remote.RelaxDefaultsDITARemoteMapsParserConfiguration"

It was designed to work when running the processing from inside Oxygen so it will probably not work when invoked from an external process. Or do you want to use if because you want to use the fix.external.references parameter?

Regards,
Radu

[vitalii_samoilenko]

Hi Radu,

I've moved Saxon libraries on top and now I'm able to run the build successfully.

Thank You for Your help!

I also tried to remove custom parser configuration and for the first look it didn't make any difference.
Since I'm new to this I'm not fully understand what impact I may have by removing it and for what reason this parameter may be used.

Could You please clarify that a bit more?

Thanks and Regards,
Vitalii.

[Radu]

Hi Vitalii,

I'm glad you got this working.
About the custom parser configuration, it's a long story. The DITA Open Toolkit publishing engine has problems publishing when there are DITA resources outside of the location where the main DITA Map is located:

https://www.oxygenxml.com/doc/versions/ ... -refs.html

so we created our own fix for it, fix which is enabled by setting the fix.external.refs.com.oxygenxml parameter for the publishing.
This fix relies on having a special parser configuration set to the publishing process but it works when running transformations from Oxygen. making this work from a command line is probably hard word. So you should probably remove the reference to that custom parser configuration class as it is not useful in your case.

Regards,
Radu

[vitalii_samoilenko]

Hi Radu,

I got Your point.

I took careful look at log and founded bunch of entries like this:

Code: Select all

...

[gen-list] [DOTJ036W][WARN] The file "C:\TestDitaIn\TestConRefs\TestConRef.xml" referenced by "C:\TestDitaIn\TestTopics\TestTopic.xml" is outside the scope of the input dita/map directory. If you do not want to see the warning message, please use the Ant parameter 'outer.control', and set the value to "quiet". Otherwise, move the referenced file "C:\TestDitaIn\TestConRefs\TestConRef.xml" into the input dita/map directory.

...

This is exactly the issue we're talking about, right?

It suggests me to move it inside the dita/map directory.
For several reasons I'm not sure that I'm permitted to do that.

Is there any impact like references would be no generated or output will miss some files?
Is there any working solution to make it fix external references via command line as it does within XML editor?

Thanks and Regards,
Vitalii.

[vitalii_samoilenko]

I've read the link carefully and founded the answers.

Please ignore my previous post.