@Prompt([OK]; “Juggling with numbers”; "@Integer( 158.92 * 100 ) / 100 = " + @Text( @Integer( 158.92 * 100 ) / 100 ) );
… and explain the results to me. Apparently, Notes thinks that Integer[ 158.92 * 100 ] / 100 equals 158.91. Did anyone else experience this strange @Integer behaviour before?
Subject: Try this …
Of course, I could use @Round( 158.92; 0.01 ) as a workaround to ensure a two-decimal number, but I still think the @Integer function should also work. Further testing shows that @Integer( 158.92 * 100 ) results in 15891. I guess it has something to do with how Notes internally handles these numbers.
Subject: RE: Try this …
You should always use @round instead of @Integer. The numbers that are stored internally in the computer are held binary, wich means that there’s allways a rounding error. So the computer remembers the binary number that’s most close to the decimal number you entered.
In this case, 158.92 is probably something like 158.9199999999997 in the computers memory.
Mulitplied by 100 it will give something like 15891.99999999998 in th computers memory.
The @Integer function just cuts off the decimal part, leaving 15891 as a result, wich is correct.
Therefore, one should never use @integer or int() for high precision calculations
Subject: Try this …
Upgrade to 6.0.2 CF1. This should be fixed now.
cheers,
Tom
Subject: RE: Try this …
I agree that it should be fixed … but it isn’t. Both 6.0.2 CF1 and 6.5 M2 show the same behaviour. Besides, our client is using 5.0.9a, but the problem is consistent in all releases (verified up to 5.0.12 and 6.5 M2).
Regards, Pascal