I put 3.14 in a Numeric Text Field using the Number Type: Double, and the Value (Double) returns 3.140000105. Why isn’t it returning exactly 3.14? I’ve even tried binding that property to round(3.14,2) and it returns the same value.
3.14 doesn’t have an exact floating point representation.
As far as the appearance in the numeric text field, you’re probably looking for the
number decimal format property.
See https://docs.python.org/3/tutorial/floatingpoint.html. The type Double is a double precision (64-bit) floating point number (not to be confused with a “DINT” or 32-bit integer in something like an Allen-Bradley PLC). Here’s an interesting calculator showing how floats are stored.
It does seem that the value reported is a 32-bit float, instead of a 64-bit float (or double precision).
Unfortunately, there is a bug here - we started using a different number formatting library that handles localization better, and in the process inadvertently ended up with some precision problems. I’ll add this forum post to that internal ticket, although it’s at a fairly low priority at the moment.
Thank you, and no worries. I found a workaround for now.
I am having a similar / same issue. Do you mind explaining what the workaround you found is?