[XSL-LIST Mailing List Archive Home]
[By Thread]
[By Date]
Re: [xsl] Floating point error
Subject: Re: [xsl] Floating point error From: David Carlisle <davidc@xxxxxxxxx> Date: Wed, 25 Jan 2006 17:42:34 GMT |
<product> <quantity>3000</quantity> <price>6.4</price> </product> (the actual quantity and price may vary, but they're always valid numbers). 6.4 can't be stored exactly as a binary floating point number (for the same reason that you can't store 1/3 as a finite-precision decimal) so in principle there is always the potential for rounding error, but given that you are just apparently looking for 4 or 5 significant figures of accuracy I'd have expected you to get the exact answer here. (I get 19200 with saxon) I assume that you are using xalan and that xalan just hands this over to the underlying JVM to do double arithmetic. xslt2 (eg saxon8) offers decimal arithmetic which can then store 6.4 exactly. not sure what to recommend as a workaround, you could try multiplying all your prices by 100 (and dividing by 100 at the end) so that it all looks like integer arithmetic which then in principle could be exact (although XSLT1 always uses floating point arithmetic, even for integers) David ________________________________________________________________________ This e-mail has been scanned for all viruses by Star. The service is powered by MessageLabs. For more information on a proactive anti-virus service working around the clock, around the globe, visit: http://www.star.net.uk ________________________________________________________________________
Current Thread |
---|
|
<- Previous | Index | Next -> |
---|---|---|
[xsl] Floating point error, Thang X. Le | Thread | [xsl] Are whitespace text nodes tri, Florent Georges |
[xsl] Floating point error, Thang X. Le | Date | [xsl] Are whitespace text nodes tri, Florent Georges |
Month |