[XSL-LIST Mailing List Archive Home] [By Thread] [By Date]

Re: [xsl] Re: XSLT 2.0 Decimal number silliness

---

---

I believe its precision

float: stores a number between +/- 1.5 X 10-45 to +/- 3.4 X 1038

The "decimal" data type should be used when you require a high level of
precision, since it can accurately store data up to 28 digits after the
decimal points (128-bit precision).


----- Original Message -----
From: "Rowland Shaw" <Rowland.Shaw@xxxxxxxxxxxxxxxxxxx>
To: <xsl-list@xxxxxxxxxxxxxxxxxxxxxx>
Sent: Monday, March 29, 2004 3:15 PM
Subject: RE: [xsl] Re: XSLT 2.0 Decimal number silliness


> Maybe I'm being really dumb here, but what is the difference between a
> decimal number and a floating point number?
>
>
> -----Original Message-----
> From: Jeff Kenton [mailto:jkenton@xxxxxxxxxxxxx]
> Sent: 29 March 2004 14:53
> To: xsl-list@xxxxxxxxxxxxxxxxxxxxxx
> Subject: Re: [xsl] Re: XSLT 2.0 Decimal number silliness
>
>
> Pete,
>
> You missed the point.  We're talking about decimal numbers here, not
floats.
>
> --jeff
>
>
> Pete Forman wrote:
> > Jeff Kenton <jkenton@xxxxxxxxxxxxx> writes:
> >
> >  > I've been looking at XSLT 2.0 decimal numbers (again), and I still
> >  > think the WC needs to do better.
> >  > [...]
> >  > <xsl:value-of select="2.0 div 3.0" />~
> >  > <xsl:value-of select="0.444 div 0.666" />~
> >  > [...]
> >  > Both lines of output should be equal.
> >
> > Why?  The majority of implementations will use a binary format such as
> > IEC 60559 to hold floating point numbers.  2.0 and 3.0 can be
> > converted exactly but the representations of 0.444 and 0.666 will be
> > inexact (0.44400000000000001 and 0.66600000000000004 in my test).  You
> > might get a better match with 0.5 div 0.75.  If you want fixed point
> > number semantics try moving the decimal points to the right before
> > dividing.
> >
> > Floating point arithmetic is inexact by definition.  Your results
> > _are_ the same to within the precision specified by the language
> > and/or implementation.

---