I have a number field and on that number field I have a custom property to get the percentage of any number entered on the field. I’m wondering how can I get the exact number when the number entered is in decimal? Like if the value entered is 16.5 and that value is divided by 100, it should output 0.165 on the custom property but its outputting 0.17. I tried using a numberFormat but it didn’t work. I tried changing the type to Float as well but it just gave me 0.170000002.
How should I format it correctly? Thank you
Change the ‘Number Type’ property to ‘Double’ and change the ‘Decimal Format’ property to something like #,##0.###, the number of #s after the decimal point are the number of decimal places that it displays it will round to. Regardless, of what shows, the value in the ‘Value (Double)’ property should be the actual value when reading/writing to that numeric text field.
Exact decimal numbers are a very, very tricky concept when talking about computers.
Decimal representations most often (especially in the world of Java) rely on 32 or 64 bit IEEE-754 encoding. This has pretty good representation across human values, but is distinctly far from exact. For instance, 0.165 has no exact 32-bit representation:
https://www.h-schmidt.net/FloatConverter/IEEE754.html
There are lots of strategies to mitigate this problem. If you strictly need to go up/down by powers of ten, then you could just artificially increase all your values by that factor (e.g. store int 16500 internally, and only insert the decimal on display).
1 Like
i was able to get this to work while on v7.8 but since we upgrade to 8.1.27 its rounding up again 