I put 3.14 in a Numeric Text Field using the Number Type: Double, and the Value (Double) returns 3.140000105. Why isn’t it returning exactly 3.14? I’ve even tried binding that property to round(3.14,2) and it returns the same value.
3.14 doesn’t have an exact floating point representation.
As far as the appearance in the numeric text field, you’re probably looking for the number decimal format property.
See https://docs.python.org/3/tutorial/floatingpoint.html. The type Double is a double precision (64-bit) floating point number (not to be confused with a “DINT” or 32-bit integer in something like an Allen-Bradley PLC). Here’s an interesting calculator showing how floats are stored.
It does seem that the value reported is a 32-bit float, instead of a 64-bit float (or double precision).
Unfortunately, there is a bug here - we started using a different number formatting library that handles localization better, and in the process inadvertently ended up with some precision problems. I’ll add this forum post to that internal ticket, although it’s at a fairly low priority at the moment.
Thank you, and no worries. I found a workaround for now.
I am having a similar / same issue. Do you mind explaining what the workaround you found is?
I assume this is no longer a pressing issue, but I chased this one down. The issue appears to have arisen in version 7.9.5 and persisted in all subsequent versions of 7.9 (.21 was the latest tested). In 8.0 this appears to have been fixed.
To clarify, a numeric input can be set to be Floating Point or Double. A floating point variable has 32-bits and a double has 64-bits. Therefore a value such as 333,333.36 will convert to 333,333.375 if the numeric input is set to Float. This is expected. If the numeric input is set to Double, this conversion should not occur.
In 7.9.5, numeric inputs set as Double would convert 333,333.36 --> 333,333.375 which is incorrect.
IGN-2520 is the internal ticket number for this issue. It has now been closed.
4 Likes