Value coming from dataset by lookup is rounded on numeric data entry component?

Ignition
, ,
1

Hello

I have a dataset tag and lookup expression on numeric entry component is taking data from this dataset. In dataset values are float like 1.8 but when I read this into a component it is rounded to 2.

lookup({[TSW_Production]Foaming/LineProductionData/TotalProductionData/ProcessParametersTable},{…/Label_1.props.text},0,0,2)

This should read 1.8 but it read 2 how can I avoid this?

Thanks

2

Does the value resolve to 2, or does it resolve to 1.8 and shows 2 in the display? If the latter, how is the numeric input’s props.format key set up?

3

it is read as 0,2 directly from the expression format is like this

0,0.##

4

This is sounding like a locale issue.

5

although after saying that, what if you set your format to 0,## ?

6

Nope does not work, still rounds it to 2. I multiple the dataset values by 100 and then divide the lookup result by 100. It is an unneccessary way of doing it but only way I could figure it out.

Thanks for the help though Jordan

7

Try changing the noMatchValue value to a fraction, lookup(tag, lookup_value, 0.0, 0, 2).

I would also recommend using the lookup and result column names instead of 0 and 2.

4 Likes
8

@mcgheeiv's got it - as the documentation notes:

The type of the value returned will always be coerced to be the same type as the noMatchValue.

With a bare 0, we have no way to know whether you're trying to specify an int, long, float, or double...so we default to integer, since it's most common. 0.0 will default to a floating point type (probably double, but I'm not sure off the top of my head). You could also do it explicitly, by wrapping your bare constant in a casting function, i.e. toDouble(0).

1 Like