[XSL-LIST Mailing List Archive Home] [By Thread] [By Date]

Re: [xsl] XSLT 2.0 Decimal number silliness

---

---

I have been watching this debate and I think it boils down to this:

The traditional way of dealing with floating point comparisons is to explicitly
use tests such as test="$a - $b < 0.1" instead of equality
comparisons. The quantities can't always be exactly equal, since the correctly
computed numerical values for $a and $b depend very much on how they are
initially specified and/or computed. You can't avoid the differences.

With the exception of integers computed using arithmetic operations that
do not involve fractions, exact equality comparison of floating point numbers
is not useful or sensible. A good tolerance value is usually based on the number
of significant digits or number of digits after the decimal point, depending
on the application.

Asking the user to specify the tolerance explicitly is very much preferable
to using tolerances by silently redefining the meaning of equality (even if the tolerance
can be user specified). This is because:

   1.  The user really does need to be aware of the issues.
   2.  The required tolerance will change from instance to instance.
   3.  The tolerance will also be needed explicitly by any software
       you are trying to interact with.
   4.   Introducing new arithmetic models for floating point computation
   hard and generally counterproductive.

Formatting on output is a different issue and I think it has been adequately
addressed.

Stan Devitt

I've been looking at XSLT 2.0 decimal numbers (again), and I still think the WC needs to do better.

[ details omitted ... ]

Using Saxon 7.x, this results in:

<?xml version="1.0" encoding="UTF-8"?>~
0.666666666666666667~
0.666666666666666666666667~

Both lines of output should be equal. There's got to be something better. At least allow specification of number of fraction digits.

--jeff

---