Numeric text field does not display what the value set to

"Float" is 32-bit. "Double" is 64-bit. Be sure to use 64bit types for entry field, named query parameter, and database column. If anything is 32-bit, it will re-introduce this loss of precision.

Also keep in mind that binary numbers cannot exactly represent fractional values, unless the fraction has a least common denominator that is a power of 2. Except for that rare case, you will always have rounding happening when converting entered decimals to binary, or converting binary back to decimal for display. Computer Science 101.

{ Doesn't matter whether a tag is involved--this is simply how floating point works in computer software, including Ignition. }

4 Likes