Is there a method to improve on the ability of Jython to calculate floating point mathematical calculations so as to not add error in the decimal place precision? I understand the basic issues with floating point calculations is how a binary system has to approximate base 10 fractions, but other than simply rounding the values up or down, is there a better way to do the math?
Use double() instead of float(). That's about it. That reduces (does not eliminate) error due to fractions that don't have a power-of-two least common denominator.
Java has some decimal data types (for things like exact currency calculations) that might be useful. You can't really pass them around, though.
A really quick search shows
- The docs seem to indicate that Jython already uses double for everything.
- Decimal might be an option, but there are performance and interop implications.
- numpy has a 128 bit float (on some machines, on some JREs)
Mostly the answer is
- determine if the loss in precision is a problem for your application. Don't spend time optimizing precision if it doesn't matter
- do the calculations in the order where rounding is done last and one time rather than at multiple steps. For example, don't calculate averages or totals from averages - do the calculation from the base numbers.
Mostly the answer is not
- stringify everything and parse the strings
And where do you get numpy for java?
Python has a Decimal module, Java has BigDecimal.
I'd bet none of this is actually necessary though.
1 Like
It likely isn't worth all that as this is for some minor inventory corrections.
But I was asked to "fix the math" and before dismissing it outright, I wanted to do a little do diligence.
You all have provided great feedback and I appreciate it!
Consider using 64-bit integers with implied decimal places.
What’s the precision on your floats, and what’s the expected range ?
When I code mostly in C I would sometimes multiply floats by a common factor to make everything integers, do the maths with that then divide back to floats.
Edit: which I guess is what Phil is suggesting ? I’m not sure…
1 Like
Likely you can't, and if you could it would require python 3. It was just a really quick search for potential options.
That won't help much if you are doing the calculations in the wrong order. And there are not any real world instruments that have the precision required to make it matter in the right order.
If the inventory is countable then use the appropriately sized integer or decimal. If the inventory is not countable (e.g., someone is integrating a flowmeter to get gallons or liters) then totalize it in something big enough.
If you need Production per Shift, Production per Day, Production per Week, and Production per Month, Production per Quarter, and Production per Year then do all of those calculations directly from the totalizer value at the beginning and end of the timespan rather than summing the shifts to make a day, summing the days to make a week, summing the weeks to make a month, summing the months to make a quarter, and summing the quarters to make a year. Round the values to the display precision when the value is displayed.
More than once I've encountered math errors when doing calculations with mixed data types such as ints and floats. Consequently, if I'm performing a calculation that involves floats or should potentially produce a fractional result, I cast all non-floats to floats before performing the calculation.
Obviously, this won't fix all floating point errors; this is just the nature of binary encoding, but this simple approach does seem to significantly reduce the severity of such problems.