Numberic Field format

I have a numeric label field for users to enter a value(rate). The value enter should be no more than 3 significant numbers. When I enter value for example .188 under the property data of the component 0.187999994 gets written to it, which will then be saved onto a database and later viewed by operators and would be miss leading.

TL/DR; Use number formatting.

Floating point numbers are binary, like anything else in a computer. Fractional numbers that have a terminating decimal representation may not have a terminating binary representation, and when such a binary number is redisplayed in decimal with all available precision, you can see that the binary number is not exactly equal to the originally entered decimal. 188/1000 has a repeating binary fraction. What you are seeing is the closest a 32-bit floating point value can get to 188/1000. If you were using 64-bit floats, there’d be more 9’s stuck in there, but it still wouldn’t (can’t) be exact.
To effectively limit the precision as you desire, you need to round to the desired decimal places anywhere the user sees the data. Ignition has number formatting patterns in all of its numeric display and entry components to provide the rounding one might want.
In programming tasks where rounding is not appropriate, like banking, special programming methods and data types are used to retain base-10 fractions. See Java’s BigDecimal data type, a common solution.

1 Like