How does the rounding in ignition work? it is not always rounding half to even or odd.
round(0.0075,3) = 0.007
round(0.0015,3) = 0.002
any explanation?
My guess is due to how numbers are stored in memory, that exactly 0.0075 and 0.0015 don't exist:
So in memory, 0.0075 is actually 0.00749..... which rounds down and 0.0015 is rounding up either due to it being half and always rounding up or rounding to the nearest even (not sure how Jython handles this).
7 Likes
That make sense.
this is what I find about this, apparently it is a common thing in computer science:
15. Floating-Point Arithmetic: Issues and Limitations — Python 3.13.5 documentation
yep
0.1 + 0.2 = 0.300000000000000004
1 Like