Hello, first of I know people have similar questions but not exactly my situation. I have a problem i have a numeric text field with no bindings, and the user just needs to input a value there. from that value, then i will execute a query to deduct from the DB. problem is that no matter what i do it always put a incorrect decimal. this is a problem as it will deduct from the db the incorrect number. I’ve tried rounding the number in a custom property, but does work, get same result. I did thy by expression language, and also thru a propertyChange Script. both situations. get the same result. as below.
Scripting Code
//this code should output to 2 decimals but it does not. it outputs
// as 19.700000763
doubles=event.source.doubleValue
round(doubles, 2)
event.source.count=doubles
another example. typing in the box 19.70 puts a doubleValue of 19.700000763. this is a problem because to this for calculations it give me incorrect results. any ideas on why? and how to fix this?
That’s the problem with using floating point values to add/subtract/sum a lot of values. Floating point isn’t accurate because there are a lot of floating point values that don’t have binary representations. If you want to properly do this then you should store your values and do your math as integers and then convert back into decimal at the end to display.
So, you would take 19.70 and multiply by 100 and then change to an integer value, then do your subtraction (the DB values should be stored as integers as well) and then at the end convert back to floating point and divide by 100. If you’re doing anything with money or weights, etc… that’s the only way to keep floating point rounding errors to a minimum.
Thank you very much. I was able to get the correct number. but got a couple questions.
many numbers in db i have as decimals(numbers coming as float, from plc(like temp, weight, amounts), also numbers as float, decimal coming from ignitionlike (qty of cases, liters, etc)
so by your recommendation I should change all that data to data type Integers? it’s a lot modifications, I have to do but, if there is a problem on doing the calculations in ignition for example:
(getting a decimal(18,2), converting that to int, calculate, the back to decimal, store in the db as decimal), but all calculations in int.
If you believe that it is not recommended. can you explain a little why? thanks in advance.
I guess it all depends on how much accuracy you really want. You could do what you were doing and change the formatting so that it only displays to the hundredths place, the value would still technically be “wrong” but no one would notice. It would take a lot of adding or subtracting until the rounding errors compounded enough to make a huge difference. But, if you’re doing banking transactions and stuff like that then fractions of a cent count so the accuracy is more important.
This is not an Ignition problem. This is how floating point numbers work in all computers. If you cannot tolerate rounding errors in arithmetic, you must use fixed-point numbering systems. Converting to integers with a number of decimals places implied is one implementation of fixed-point arithmetic. This approach is common in the controls world because ints and floats are generally all there is available in PLCs.