Request service 4 creating syntax validation code

[jfontein]

Hello,

We have a custom application being developed by a third party. They created an XML syntax for us to customize certain parts of that application.
We want to edit our XML files with Oxygen. However, Oxygen doesn't recognize all of our syntax and therefore can't validate it completely and does other strange things with our XML-tags (auto suggest things) we don't want.
We're looking for someone who can write some 'code' and helps with Oxygen settings so that Oxygen helps us create and check our XML-files easily.

We ourselves are no real IT-people and just understand our 'own' XML-syntax (and like to keep it at that level).

I included a sample (composed of different snippets) of our XML syntax to give you an idea. The sample contains the majority of the syntax applicable.

Please let us know if you are interested to take on this job and give a rought estimate on time and costs.

Joost Fontein

Code: Select all

<ELEMENTINFO_PATTERN function="TAG_F_CAR" type="TAG_PAVING" icon="Files/Images/Elements/FunctionIcons/fi_car.png">

	<PE_CM_ElementInfo productfamily="paving" function="car" elementtrafficclass="heavy">



<VS UID="VAR_OLD_PAVING_TYPE"/>

<VS UID="VAR_PAVING_TRAFFIC_CLASS"/>

<VS UID="VAR_PAVING_CATEGORY"/>



		<INCLUDE UID="11**_INCL_10" file="1910_CopyOldElementCheck.ElementInclude"/>

		<INCLUDE UID="11**_INCL_20" file="1200_Paving.ElementInclude"/>



<INCLUDE UID="1200_INCL_10" file="1900_VariablesPaving.ElementInclude"/>



<IF UID="1200_IF_10" exp="$Get('COMPOSED')">

	<FB UID="UID_COMPOSED_FREQUENCY" label="TAG_COMPOSED_FREQUENCY" type="AKFloat" value="1"/>

</IF>



<FB UID="UID_RELATIVE_VERTICAL_OFFSET" label="TAG_PAVING_RELATIVE_VERTICAL_OFFSET" type="AKFloat" value="0.0"/>



<TXT UID="UID_DEVIDER_PAVING" label="TAG_DEVIDER_DASHED"  />



<LB UID="UID_PAVING_TYPE" label="TAG_PAVING_TYPE" type="AKString">

	<LBO label="TAG_LBO_NULL"/>

	<LBO label="TAG_PAVING_TYPE_ASPHALT"/>

	<LBO label="TAG_PAVING_TYPE_POURED_CONCRETE"/>

	<LBO label="TAG_PAVING_TYPE_OPEN_PAVING"/>

	<!-- JF concrete plates are a specific form of OPEN PAVING, but is mentioned here as if a seperate PAVING TYPE -->

	<LBO label="TAG_PAVING_TYPE_CONCRETE_PLATES"/>

</LB>



<IF UID="1210_IF_20" 

	exp="$ Get('OLD')

			||	( Get('NEW') 

					&& (!Get('LB','UID_ASPHALT_RENEWAL'):{'TAG_LBO_NULL'}) 

				)

	">

	<LB UID="UID_PAVING_TRAFFIC_CLASS_ASPHALT" label="TAG_PAVING_TRAFFIC_CLASS" type="AKString" copyselected="VS.VAR_PAVING_TRAFFIC_CLASS" >

		<!-- starting-point: if section.TrafficClass==light this implies that for all elements in that section elementtrafficclass==light is applicable -->

		<LBO label="TAG_PAVING_TRAFFIC_CLASS_LIGHT"/>

		<LBO label="TAG_PAVING_TRAFFIC_CLASS_BASIC" 

			default="$

				(	GetSection('UID_SECTION_TRAFFIC_CLASS','Value'):{'TAG_SECTION_TRAFFIC_CLASS_BASIC'}

					&& Get('ElementInfo','elementtrafficclass'):{'basic'||'heavy'}

				)

				||(	GetSection('UID_SECTION_TRAFFIC_CLASS','Value'):{'TAG_SECTION_TRAFFIC_CLASS_HEAVY'}

					&& Get('ElementInfo','elementtrafficclass'):{'basic'}

				)

		"/>

		<LBO label="TAG_PAVING_TRAFFIC_CLASS_HEAVY" 

			filter="$Get('ElementInfo','elementtrafficclass'):{'heavy'}"

			default="$

				GetSection('UID_SECTION_TRAFFIC_CLASS','Value'):{'TAG_SECTION_TRAFFIC_CLASS_HEAVY'}

				&& Get('ElementInfo','elementtrafficclass'):{'heavy'}

		"/>

		<LBO label="TAG_LBO_CUSTOM_HEIGHT"/>

		<LBO label="TAG_LBO_UNKNOWN" filter="$Get('OLD')"/>

	</LB>

	<IF UID="1210_IF_20_10" exp="$(!Get('LB','UID_ASPHALT_RENEWAL'):{'TAG_ASPHALT_RENEWAL_SURF_ONLY'||'TAG_ASPHALT_RENEWAL_POSTPONE_SURF'})">

		<FB UID="UID_PAVING_THICKNESS_ASPHALT" label="TAG_PAVING_THICKNESS" type="AKFloat"/>

	</IF>

	<IF UID="1210_IF_20_30" exp="$Get('LB','UID_ASPHALT_RENEWAL'):{'TAG_ASPHALT_RENEWAL_SURF_ONLY'}">

		<FB UID="UID_PAVING_THICKNESS_ASPHALT_SURF_ONLY" label="TAG_PAVING_THICKNESS_ASPHALT_SURF_ONLY" type="AKFloat"/>

	</IF>

	<IF UID="1210_IF_20_40" exp="$Get('LB','UID_ASPHALT_RENEWAL'):{'TAG_ASPHALT_RENEWAL_POSTPONE_SURF'}">

		<FB UID="UID_PAVING_THICKNESS_ASPHALT_POSTPONE_SURF" label="TAG_PAVING_THICKNESS_ASPHALT_POSTPONE_SURF" type="AKFloat"/>

	</IF>



</IF>



		<RBG UID="UID_BASE_ADJUSTMENT" label="TAG_BASE_ADJUSTMENT" type="AKString" tooltip="TAG_TT_BASE_ADJUSTMENT">

			<RBO label="TAG_ALWAYS_KEEP_BASE" tooltip="TAG_TT_ALWAYS_KEEP_BASE"/>

			<RBO label="TAG_ALWAYS_RENEW_BASE" tooltip="TAG_TT_ALWAYS_RENEW_BASE"/>

			<RBO label="TAG_DETERMINE_BASE_ADJUSTMENT_AUTOMATICALLY" tooltip="TAG_TT_DETERMINE_BASE_ADJUSTMENT_AUTOMATICALLY"/>

		</RBG>



	<CB UID="UID_REINFORCED_ASPHALT" label="TAG_REINFORCED_ASPHALT" type="AKBoolean" value="false"/>





	</PE_CM_ElementInfo>

</ELEMENTINFO_PATTERN>

[Radu]

Dear Joost,

The Oxygen XML Editor development team does not provide services for XML customizations.
We have a special section on our forum where you can see contact email address for companies who provide different XML-related services:
http://www.oxygenxml.com/forum/forum10.html

In the meantime, if you want to make an attempt yourself:

The XML that you posted is not well formed XML.
It is illegal in XML to use the & character inside attribute (and element) values. So in your case all occurrences of the && will have to be replaced with the character entities &&.
So these errors need to be corrected as no XML-aware parser would be able to parse an XML which is not well-formed.

After you do this, if you have the XML opened in Oxygen you can go to the Document main menu to the XML Document submenu and choose the Save Structure action. Oxygen will try to auto-detect the structure of the XML file and then you will be able to save a DTD containing the structure representation.

If you associate such a DTD with the XML for example by specifying the document type definition on top of the XML file like:

<!DOCTYPE ELEMENTINFO_PATTERN SYSTEM "dtdFileName.dtd">

then Oxygen should give you content completion and validation based on it.

Oxygen also has support for easily adding new file template (for creation of XML files) to the New dialog wizard:

http://www.oxygenxml.com/doc/ug-oxygen/ ... og-sa.html

Regards,
Radu

[jfontein]

Gave it a little try, but run into too many questions. That's what we would need help for.

[sorin_ristache]

Hello,

We can go over your questions one by one if you want and if they are related with Oxygen features or with editing XML files in Oxygen or if Oxygen can assist you in correcting / processing your XML files.

First you have to make the file that you have a well-formed XML document as Radu explained. If you have a file that contains some XML tags but does not conform to the requirements for XML well-formed docuemnts from the XML specification then you cannot use that file as an XML document. That means it cannot be parsed by any XML parser, it cannot be validated against a schema for XML documents (DTD, W3C XML Shema, etc), it cannot be transformed in an XSLT transformation, etc.


Regards,
Sorin

[jfontein]

the document should be parsed as xml, but for our purposes we need to parse the code between the " commas using a regular expression (as far as the XML is concerned that is a value). So that would mean the && could remain there (!?).

Some of the issues we have:
- Our syntax <INCLUDE UID="UID" file="File.ElementInclude"/>
- When I would enter <INCLUDE... it Oxygen automatically gives me <INCLUDE></INCLUDE>
- Same with other tags
- The code between " @ exp="", filter="" and default="" would need to be coloured using regular expressions.

[jfontein]

It should also be possible to directly FTP from Oxygen ?!

[Radu]

Hi,

So:

the document should be parsed as xml, but for our purposes we need to parse the code between the " commas using a regular expression (as far as the XML is concerned that is a value). So that would mean the && could remain there (!?).

No, this would mean not well formed XML, there is no customization which would make Oxygen get over the problem.

Some of the issues we have:
- Our syntax <INCLUDE UID="UID" file="File.ElementInclude"/>
- When I would enter <INCLUDE... it Oxygen automatically gives me <INCLUDE></INCLUDE>
- Same with other tags

If you create the DTD grammar from the XML document as I explained in the previous post and edit it you can set the content model for such tags to be empty. So instead of:
<!ELEMENT INCLUDE (#PCDATA)>

you would edit it like:

<!ELEMENT INCLUDE EMPTY>

When such elements would be added by content completion they would be added by Oxygen in the collapsed form like <INCLUDE/>.

- The code between " @ exp="", filter="" and default="" would need to be coloured using regular expressions.

We do not have customizations which would render different syntax highlight colors for certain attribute values. So this cannot be achieved with Oxygen.

It should also be possible to directly FTP from Oxygen ?!

Yes, sure, from the File menu you have the Open URL... action for this.
Another alternative:
In the Oxygen Preferences->Data Sources you can configure a new FTP connection which will be shown in the Data Source Explorer view.

Regards,
Radu

[jfontein]

OK. Did that and 'kinda works'...

- The .dtd file ONLY contains the specs for the <INCLUDE/> tag (which I manually changed as you specified.
- Why aren't there the specs for all other tags e.g.:
- IF
- CM
- LB
- LBO
which all have consistent syntax in the file I used?


- Would it be possible to have my files use the .dtd specs based on the extension of my files? I use .Element .ElementInclude and some other specific extensions.

[jfontein]

This is the code on which I created the .dtd

Code: Select all

<INCLUDE UID="1200_INCL_10" file="1900_VariablesPaving.ElementInclude"/>

<IF UID="1200_IF_10" exp="$Get('COMPOSED')">

	<FB UID="UID_COMPOSED_FREQUENCY" label="TAG_COMPOSED_FREQUENCY" type="AKFloat" value="1"/>

</IF>



<FB UID="UID_RELATIVE_VERTICAL_OFFSET" label="TAG_PAVING_RELATIVE_VERTICAL_OFFSET" type="AKFloat" value="0.0"/>



<TXT UID="UID_DEVIDER_PAVING" label="TAG_DEVIDER_DASHED"  />



<LB UID="UID_PAVING_TYPE" label="TAG_PAVING_TYPE" type="AKString">

	<LBO label="TAG_LBO_NULL"/>

	<LBO label="TAG_PAVING_TYPE_ASPHALT"/>

	<LBO label="TAG_PAVING_TYPE_POURED_CONCRETE"/>

	<LBO label="TAG_PAVING_TYPE_OPEN_PAVING"/>

	<!-- JF concrete plates are a specific form of OPEN PAVING, but is mentioned here as if a seperate PAVING TYPE -->

	<LBO label="TAG_PAVING_TYPE_CONCRETE_PLATES"/>

</LB>



<IF UID="1200_IF_15" exp="$Get('OLD')">

	<CM UID="1200_IF_15_CM_10" exp="VS.VAR_OLD_PAVING_TYPE.Value=LB.UID_PAVING_TYPE.Value"/>

</IF>



<IF UID="1200_IF_20" exp="$Get('LB','UID_PAVING_TYPE'):{'TAG_LBO_NULL'}">

	<CM UID="1200_IF_20_CM_10" exp="VS.UID_BACKGROUND_URL='NULL'"/>

	<CM UID="1200_IF_20_CM_20" exp="VS.VAR_PAVING_CATEGORY='NULL'"/>

</IF>





<IF UID="1200_IF_30" exp="$Get('LB','UID_PAVING_TYPE'):{'TAG_PAVING_TYPE_ASPHALT'}">

	<CM UID="1200_IF_30_CM_10" exp="VS.VAR_PAVING_CATEGORY='CLOSED'"/>

	<INCLUDE UID="1200_IF_30_INCL_10" file="1210_Asphalt.ElementInclude"/>

	<INCLUDE UID="1200_IF_30_INCL_20" file="1410_BackgroundAsphalt.ElementInclude"/>

</IF>



<IF UID="1200_IF_40" exp="$Get('LB','UID_PAVING_TYPE'):{'TAG_PAVING_TYPE_POURED_CONCRETE'}">

	<CM UID="1200_IF_40_CM_10" exp="VS.VAR_PAVING_CATEGORY='CLOSED'"/>

	<INCLUDE UID="1200_IF_40_INCL_10" file="1215_PouredConcrete.ElementInclude"/>

</IF>



<IF UID="1200_IF_50" exp="$Get('LB','UID_PAVING_TYPE'):{'TAG_PAVING_TYPE_OPEN_PAVING'||'TAG_PAVING_TYPE_CONCRETE_PLATES'}">

	<CM UID="1200_IF_50_CM_10" exp="VS.VAR_PAVING_CATEGORY='OPEN'"/>

	<INCLUDE UID="1200_IF_50_INCL_10" file="1220_OpenPaving.ElementInclude"/>

	<INCLUDE UID="1200_IF_50_INCL_20" file="1420_BackgroundOpenPaving.ElementInclude"/>

</IF>



<INCLUDE UID="1200_INCL_20" file="1230_TemporaryPaving.ElementInclude"/>



<INCLUDE UID="1200_INCL_30" file="1310_Base.ElementInclude"/>



<IF UID="1200_IF_60" exp="$Get('OLD')">

	<INCLUDE UID="1200_IF_60_INCL_10" subs="%OLDorNEW%=OLD" file="1320_Subbase.ElementInclude"/>

</IF>

<IF UID="1200_IF_70" exp="$Get('NEW')">

	<INCLUDE UID="1200_IF_70_INCL_10" subs="%OLDorNEW%=NEW" file="1320_Subbase.ElementInclude"/>

</IF>



<INCLUDE UID="1200_INCL_40" file="1605_DimensioningPavingThickness.ElementInclude"/>

<INCLUDE UID="1200_INCL_50" file="1610_DimensioningBaseThickness.ElementInclude"/>

<INCLUDE UID="1200_INCL_60" file="1615_DimensioningSubbaseThickness.ElementInclude"/>



<CS UID="CROSS_SECTION" hidden="yes">

	<INCLUDE UID="1200_INCL_70" file="1510_BackgroundCrossSectionPaving.ElementInclude"/>

	<INCLUDE UID="1200_INCL_80" file="1630_DimensioningCrossSectionPaving.ElementInclude"/>	

</CS>

[Radu]

Hi,

- The .dtd file ONLY contains the specs for the <INCLUDE/> tag (which I manually changed as you specified.
- Why aren't there the specs for all other tags e.g.:
- IF
- CM
- LB
- LBO
which all have consistent syntax in the file I used?

This is because the XML which you used to generate the DTD is not well formed (XML should have only a single root element containing all other elements inside). The XML which you posted does not have a single root element.

Would it be possible to have my files use the .dtd specs based on the extension of my files? I use .Element .ElementInclude and some other specific extensions.

Yes, in the Oxygen Preferences->Document Type Association page you can create a new document type association which defines a new association rule based on the file name like *.Element, *.ElementInclude and having all other fields default to *.
Then in the Schema tab you can define the default DTD associated with the document by selecting the Schema Type and Schema URI.

Regards,
Radu

[jfontein]

OK. Thanks! Seem to get somewhere!

I created that DocumentTypeAssociation, but it seems that I still need to paste that <!DOCTYPE ELEMENTINFO_PATTERN SYSTEM "dtdFileName.dtd"> on top of each XML (*.Element) file ?!?!
- Thought that wouldn't be necessary as I created that DocTypeAssociation...???

By the way: I didn't change the && and all seems to be OK... It just handles the && as part of the whole string in exp="blah && blah"

Code: Select all

<IF UID="1910_IF_COPY_FROM_OLD_ELEMENT_NEW" exp="$Get('NEW') && Get('VS','VAR_COPY_FROM_OLD_ELEMENT'):{'TRUE'}">

   <BLAH></BLAH>

</IF>

[/i]

[Radu]

Hi,

There was probably a mistake in creating the document type association, I'll try to give you some screenshots.

First, there is a view which can be shown in Oxygen called Properties which shows for each XML file the associated document type. This is useful to see that the association is made.



The association rules in the document type for matching a file name like "testPath" should be like:



And the schema association tab should refer to the DTD like:

By the way: I didn't change the && and all seems to be OK... It just handles the && as part of the whole string in exp="blah && blah"

XML with such & which are not escaped to & either in attribute values or in text is not wellformed and Oxygen will signal this with a validation error:

The entity name must immediately follow the '&' in the entity reference.

Regards,
Radu

[jfontein]

Hi Radu,

Got it up and running! Thanks for that!

A few more questions to make it work even more smooth, based on what I found in http://www.xmlfiles.com/dtd/dtd_attributes.asp (and it's pervious page):

- In the project we have multiple files. All elements get a unique ID, where they should be unique across all those files. Is there a way to check on that using the DTD and/or Oxygen settings?

- It should be possible to limit the options for the values of an attribute. If I understand the documentation well it should be something like:

Code: Select all

<!ELEMENT OPTION_PATTERN (#PCDATA | CM | IF | INCLUDE | VB | VF | VS | CB | FB | LB | RBG | TB | TXT)*>

    <!ATTLIST OPTION_PATTERN function  (function1|function2) "function1">

But this doesn't seem to work... Any hint on that?

- It should also be possible to specify the number of child elements, using * + ? (see the previous page of the url). But this doesn't seem to work either. (Get error message when I remove the *).

Code: Select all

<!ELEMENT LAYERS (#PCDATA | LAYER)*>



    <!ELEMENT LAYER EMPTY>

        <!ATTLIST LAYER TAG_NAME CDATA #REQUIRED>

        <!ATTLIST LAYER location CDATA #REQUIRED>

Any hint on that?


Regarding the &&. I think I'll sustain with just ignoring that error message.

Regarding the regular expressions to check our 'custom' code that is now part of the value of the 'exp', 'default' and 'filter' attribute in some of our elements: would it be possible with Oxygen to write a colouring schema that checks that document as-if it is not an XML document? The first part of the regular expression should than only check the text between "" after it found 'exp=', 'default=' or 'filter='. I could then check my complete project in 2 (manual) runs:
1) check XML-coding with the DTD I created
2) check the 'custom code' using the regular expressions
Could that be a 'work around'?

[Radu]

Hi,

In the project we have multiple files. All elements get a unique ID, where they should be unique across all those files. Is there a way to check on that using the DTD and/or Oxygen settings?

You could probably implement some Schematron checks for this if you know exactly where the XML files are located on disk. See more about Schematron at the end of my post. But you need to be familiar with writing XPath 2.0 expressions.

The Schematron would look like:

Code: Select all

<?xml version="1.0" encoding="UTF-8"?>

<schema xmlns="http://purl.oclc.org/dsdl/schematron" queryBinding="xslt2">

    <pattern id="checkIDs">

        <!--For each element which contains an UID-->

        <rule context="*[@UID]">

         <!--Gather all XML files from the current directory-->

            <let name="allIds" value="collection('./?select=*.xml')//@UID"/>

            <let name="id" value="@UID"/>

          <!--Show an error if an id value is encountered more than once-->

            <report test="sum(for $i in $allIds return if ($id = $i) then 1 else 0) > 1">

                The attribute value <value-of select="$id"/> is duplicate.

            </report>

        </rule>

    </pattern>

</schema>

It should be possible to limit the options for the values of an attribute. If I understand the documentation well it should be something like:.....

In the DTD I specified that the function attribute of the PE_CM_ElementInfo can have the two values:

Code: Select all

<!ATTLIST PE_CM_ElementInfo function  (function1|function2) "function1">

And content completion for the value worked in my case when editing the XML.

It should also be possible to specify the number of child elements, using * + ? (see the previous page of the url). But this doesn't seem to work either. (Get error message when I remove the *)......

Yes, it should, but DTD has a limitation to express this when an element is declared as having mixed content:

http://www.featureblend.com/dtd-element ... ntent.html

So you can use constructs like:

Code: Select all

<!ELEMENT LAYERS (LAYER+)>

or

Code: Select all

<!ELEMENT LAYERS (LAYER?)>

but not like this:

Code: Select all

<!ELEMENT LAYERS (#PCDATA | LAYER)+>

#PCDATA means in this context that the element would also accept text content (non-whitespace text content). So if you do not need to insert text content directly inside the element just remove the #PCDATA.

Regarding the regular expressions to check our 'custom' code that is now part of the value of the 'exp', 'default' and 'filter' attribute in some of our elements: would it be possible with Oxygen to write a colouring schema that checks that document as-if it is not an XML document? The first part of the regular expression should than only check the text between "" after it found 'exp=', 'default=' or 'filter='. I could then check my complete project in 2 (manual) runs:
1) check XML-coding with the DTD I created
2) check the 'custom code' using the regular expressions
Could that be a 'work around'?

What do you mean by check? Validation or special colors in the "special" areas?
Oxygen does not have the support to offer special coloring depending on certain areas of the file.

For validation of those areas using regular expressions you could use some Schematron rules. Schematron is another type of XML validation standard which can give you enhanced validation. I will not go into details here, you can google it for more details if you are interested in the approach. Schematron is based on XSLT XPath expressions which can be used to check validity in a certain context.

For example you create a file called test.sch with the following content:

Code: Select all

<?xml version="1.0" encoding="UTF-8"?>

<schema xmlns="http://purl.oclc.org/dsdl/schematron" queryBinding="xslt2">

    <pattern id="checkIDs">

        <rule context="*[@exp]">

            <report test="matches(., '(.*)')">

                The expression does not match the regular expression.

            </report>

        </rule>

    </pattern>

</schema>

This schematron file will check that the value of each "exp" attribute value matches a Regular Expression (in this case (.*)).

So if at the top of the XML file you have a reference to the schematron file:

Code: Select all

<!DOCTYPE ELEMENTINFO_PATTERN SYSTEM "test.dtd">

<?xml-model href="test.sch" type="application/xml" schematypens="http://purl.oclc.org/dsdl/schematron"?>

<ELEMENTINFO_PATTERN function="TAG_F_CAR" type="TAG_PAVING" icon="Files/Images/Elements/FunctionIcons/fi_car.png">

...........

then the XML will be validated with the Schematron.

VERY IMPORTANT: the XML must be wellformed in order to properly process it with the Schematron so you MUST escape the & characters in order to make it work.

Regards,
Radu

[jfontein]

Thanks for the update!

- I'll remove that #PCDATA. Don't need that.
- I'll leave that schematron stuff for now. Maybe get back to that later, but will let someone else do that. That's realy beyond my skills.

Think that's it for now. This will already help us a lot.

Regards,
Joost