Getting required precision from named query and event handler

Hello,

I have a button that I am triggering a named query from. Here is the script:

params = {
	"HardwareIDParam": int(event.source.parent.getComponent('BagIDLabel').text),
	"QuantityParam": event.source.parent.getComponent('QuantityLabel').value
		}

system.db.runNamedQuery("INSERT Inventory Transaction", params)

Everything works, except when I run a select query on my InventoryTransaction table, the quantity is showing as a rounded number instead of as a float with one decimal precision which is what I require. The value of the QuantityLabel is coming from a Vision Client Tag that I created of float data type. This tag is getting its value from an input on another screen. I set the datatype in the table as float as well.

Not sure what is happening here. Any guidance would be greatly appreciated. Thanks!

If you can share your SQL table design and the queries, it would help.

1 Like

That's not really how floating point precision works.

E.X, there's no such thing as 0.2, only 0.200000000000000011102, inside the computer - and the margin of error is going to change based on how far away from zero your values are.

That probably isn't your problem here, because computers are good at lying to you about the apparent precision of numbers precisely because of this fact, but it's important to recognize. If you are dealing with, say, currency, where exact decimal math is required, any floating point type is inadequate and not fit for purpose.

Hi David,

I feel a little bit silly here.

In looking up my named query so I could type a response for you, I realized that I had my quantity parameter set to a data type of Int instead of Float. I simply changed the data type and it works like I need it to :man_facepalming:

I do think you get credit for the solution, though, since it was your question that prompted me to look closer.

Thanks!

1 Like

Hi PGriffith,

This is really cool info. Didn't help me with my problem at all, but a fun little tidbit.

Thanks!

1 Like

I was wondering what data types you had set up, and was hoping when you shared the information it would be indicative of the problem, looks like you found what I was going to look for, well done.

3 Likes