stdDev is not correct

Hi,

I found stdDev expression function is not conform Ignition manual as pictured. I try to calculate this table on excel. And value is correct and conform Ignition manual. Could please check or suggest us to calculate stdDev?


stdDev2

Thank you,

Thanks for catching that.

There are two kinds of standard deviation, population and sample. The 4.0053 answer is for population standard deviation and 3.4687 is for sample standard deviation. For most users, sample standard deviation is more appropriate.

Thank you for prompt response.

As you mentioned before, Is Ignition use sample standard deviation? Could you please double check for this?

stDev2
stDev

STDEV (n-1): STDEV function - Office Support

STDEV.P (n): STDEV.P function - Office Support

Yes, we’re using sample standard deviation. As calculated by:

                public Double calculate(Dataset ds, int col) {
                    Double mean = super.calculate(ds, col);
                    if (mean == null) {
                        return null;
                    }
                    // std dev is the sqrt((sum of (x_i-mean)^2)/count)
                    int count = ds.getRowCount();
                    int realCount = 0;
                    double sum = 0;
                    for (int r = 0; r < count; r++) {
                        Number num = (Number) ds.getValueAt(r, col);
                        if (num != null) {
                            sum += Math.pow(num.doubleValue() - mean, 2);
                            realCount++;
                        }
                    }
                    if (realCount == 0) {
                        return null;
                    } else {
                        sum = sum / realCount;
                        return Math.sqrt(sum);
                    }
                }

// std dev is the sqrt((sum of (x_i-mean)^2)/count) >> This is population stadard deviation. Is sample standard deviation /count-1 as pictured before?

https://www.calculator.net/standard-deviation-calculator.html



1 Like

Hi Kathy,

is it not the other way round?
Sample standard deviation should be 4.0053 and Population standard deviation 3.4687

I’m facing the same issue when I’m using the stddev() function, returning me the population one. (version 7.9.2)

Thank you.

Valerio

1 Like

You actually can’t know that.

For some data, you have the complete population (f.e. when weighing bags, all bags of a production run are weighed and you can calculate the standard deviation on that run). For other data, you will be forced to sample (f.e. sampling a continious temperature measurement during an operation).

That said, I’ve never liked how statistics always seems to have annoying definitions like these, with seemingly random constants.

3 Likes

I agree, having both population and sample standard deviation functions available would be nice. It should be up to the user to determine which is useful.

You might find this interesting for your background knowledge: https://en.wikipedia.org/wiki/Bessel's_correction

2 Likes