Standard Deviation issue

General Discussion
1

I have a table with the following numbers

4.577
4.578
4.578
4.578
4.578
4.578
4.579
4.580
4.579
4.579
4.580
4.578
4.581
4.582
4.582
4.581
4.581
4.582
4.581
4.571
4.569

If I use the StDev(tablename,data,“numbers”)

I get this number

0.00321208

If I use a sql statement

I get this number
0.00339408

When I use minitab or excel I get this number

0.003291403

any ideas why its so different?

2

Without spending a lot of time digging, a couple of possibilities come to mind.

First is rounding. Are those numbers exactly 3 decimal places, or being displayed as three decimal places? Obviously any calculation using the full number of decimal places can be different than a calculation using a rounded number.

Also, there are two types of standard deviation, sample and population, and their results will be different. Ignition’s stdDev expression function uses sample standard deviation.

3

For grins, I plugged the data you posted into Wolfram Alpha. I got 0.00321208 as the sample std dev and 0.0032914 as the population std dev. No idea how sql is getting its number unless rounding is coming into play.

1 Like
4

I’m rounding all the numbers to 3 decimal places. Thank you for your response