Setpoint - Decimal places and Error out of bounds

[awybenga]

I have some setpoint tags with for example Engineering limits set 0.4 to 0.9. I want it to error on out of bounds (for example if the operator enters 5 instead of 0.5 I don't want it forcing a 0.9.). Sometimes I cannot enter a setpoint of 0.9. It says its out of bounds.

What part of the tag or component can I set to ensure it is not being entered as 0.9000001 etc. The operators need to be enter the max limit.

Any advice or fixes I could try?

Thanks

[pturmel]

This is almost certainly due to floating point fraction behaviors, since 0.9 doesn't have a precise binary representation. (Repeats in binary, much like ⅓ repeats as a decimal.)

Possibly exacerbated by this related phenomenon:

Numeric Text Field - Float vs Double Issue

This is a known/accepted issue in Vision's numeric text field component; if you want to talk to support you can reference our internal ticket number IGN-11528. We're delegating the internal string -> number type to an external library, and that external library goes from smallest -> largest possible presentation, and 13.3 happens to be a value that can be (acceptably to the library, at least) represented in a 32-bit float, and therefore that's what is returned. Changing that logic in the numer…