I came across a rounding error when I was running an XQuery.
Here is the error:
xquery version "1.0";
number(3.1) + number(3.2)
which returned:
6.300000000000001
Not the expected value of 6.3. Note that the XQuery function number() returns double precision data.
After a note from Eric Bruchez he suggested I cast the numbers to decimal:
xquery version "1.0";
xs:decimal(3.1) + xs:decimal(3.2)
and the problem seems to go away. He also showed that he could reproduce this error in other Java JVM languages like Scala.
Let me know if anyone else has seen this problem before and has any other suggestions for a fix.
Thanks! - Dan
