Oh, nice find!
Also, depending on your needs, Apache Math is worth looking at.
I didnāt really find JScience myself. I learned about it when contributing to the now-defunct CompPad add-on for OpenOffice/LibreOffice.
I came across this thread when looking for options on how to add some statistical math functions to ignition. I followed the Apache Math link, and it looks like thereās everything I could want there. However, Iām not sure how to go about adding the functionality to ignition.
Is this something that you need to make a module with? If so, how do you do that? Iām a bit surprised that someone hasnāt created a Math module, or ignition hasnāt added the functions to the core software, given the frequency with which questions regarding numpy are asked. Of course, Iām guessing my perception is a bit skewed after searching the forums for a while on this topic.
For what itās worth, at the moment, Iām interested in mode, median, standard deviation, kurtosis, and skewness. I suppose itās possible Iād be looking for more down the road sometime, but those would get me started.
If youāre willing to have gateway only access, you donāt need a module, but a module will absolutely be the easiest way to get access to Apache Math.
Honestly, the things youāre looking for are really easy to calculate in your own script module without Apache Math, especially if you add some functions to calculate the nth central moment (which, with your standard deviation calc, will give you kurtosis and skewness). And, of course, mean and std dev are already available in the expression language.
But adding those kind of math functions is a great idea, and you should suggest it at ideas.inductiveautomation.com. This is a great kind of project for me to give to new developers.
Yeah, I will probably try to code my own functions that Iām looking for. Iām just a little surprised that those kind of functions arenāt available already. I suppose data analysis like that might be a little outside the realm of traditional SCADA systems, but Ignition is so good at working with databases and scripting, it just makes sense to do it internally.
I added the idea. Hereās the link in case anyone else reading this wants to upvote.
Thanks!
I'd be hurt that no one mentioned this... if it wasn't for my huge ego...
I actually saw that post when I was looking around! However, itās missing several of the functions I was looking for. And, worse, when I tried uploading my project to my 7.9.6 gateway, it complained that [global] was already existing on the gateway and let me rename it. I did, but it appeared to be an empty project. I couldnāt find any scripting assets in the project. I didnāt pursue it further as it didntā appear to have everything I needed anyway.
Import it by right-clicking on āGlobalā at the top of the tree and select āImportā. Afterward, you should see it in the shared script library.
That being said, Iām always open to new functionality. I had originally hoped that it would be a collaborative effort, but so far itās been a team of one. Heh.
Ohhhhh, thatās how you import that! Iāll have to give it a try!
No worries, Brian!
Like I said I donāt mind working on new functionality. I just need to know what functionality to work on.
Below is a mode function I modified from an example I found on the web. I added it to your math script and it seems to be working. Give it a try and see what you think. Iām sure thereās other ways to be doing mode calculations (Iām no mathematician), but I think this will serve my purposes.
Probably the more proper thing to do instead of simple rounding for real values would be to calculate intervals to assign the data points to and then calculate a mode from that.
# Function to return all modes, this function takes into account multimodal distributions.
# Optional digits argument will round numbers to the specified decimal places.
# The rounding function is incompatible with string types.
# Function returns all modes as list or an empty list if such value doesn't exist.
def mode(l,digits=None):
if len(l) > 1: #
#Creates dictionary of values in l and their count
d = {}
for value in l:
if digits != None:
value = round(value,digits)
if value not in d:
d[value] = 1
else:
d[value] += 1
if len(d) == 1:
return [value]
else:
# Finds most common value
i = 0
for value in d:
if i < d[value]:
i = d[value]
# All values with greatest number of occurrences can be a mode if:
# other values with less number of occurrences exist
modes = []
counter = 0
for value in d:
if d[value] == i:
mode = (value, i)
modes.append(mode)
counter += mode[1] # Create the counter that sums the number of most common occurrences
# Example [1, 2, 2, 3, 3]
# 2 appears twice, 3 appears twice, [2, 3] are a mode
# because sum of counter for them: 2+2 != 5
if counter != len(l):
return [mode[0] for mode in modes]
else:
return []
else:
return []
Some example results from the function.
>>> print shared.math.mode([1])
[]
>>> print shared.math.mode([1, 1])
[1]
>>> print shared.math.mode([1, 1, 2, 2])
[]
>>> print shared.math.mode([1, 2, 3, 4, 5])
[]
>>> print shared.math.mode([1, 1, 2, 2, 3, 3, 4])
[1, 2, 3]
>>> print shared.math.mode([1, 2, 3, 4, 4])
[4]
>>> print shared.math.mode(['string', 'string', 1])
['string']
>>> print shared.math.mode(['string', 'string', 1, 1, 1])
[1]
>>> print shared.math.mode([0.57099998,0.571367845,0.571367845,0.571657243,0.571657243],3)
[0.571]
Iām a bit confused on your outputā¦
>>> print shared.math.mode([1, 1, 2, 2])
[]
gives an empty return set, but
[1, 2, 3]```
does not? It doesn't seem consistent.
I was a little confused about that myself. Thereās a comment in the code:
# All values with greatest number of occurrences can be a mode if:
# other values with less number of occurrences exist
I havenāt done research on how modes should be handled with uniform distributions (like I said, not a mathematician). I just assumed that it was some edge case the programmer was handling. Itās one of the reasons I posted it to be picked apart
Edit: Interesting reference I found
Definitely agree with the āno consensus partā. LOL
Brings up a good thought for someone specāing a module (which may end up being me)ā¦ probably the best user experience is to allow an optional param to specify how this situation is handled. Because no matter how you handle it, someone will complain.
Hereās what I came up with. include optional parameter to allow all modes vs. no mode for even distributions.
def mode(listIn, all_modes = 0):
bins = list(set(listIn))
counts = [listIn.count(x) for x in set(listIn)]
maxCounts = max(counts)
indices = [i for i, x in enumerate(counts) if x == maxCounts]
dataOut = []
if all_modes == 0:
if len(set(counts)) != 1:
for i in indices:
dataOut.append(bins[i])
else:
for i in indices:
dataOut.append(bins[i])
return dataOut
For what itās worth, Iām modifying a statistics package that you may be interested in. More details coming.
Awesome! I havenāt had any time to play with this stuff today. Other duties and stuffā¦
If I could make a request, please include in the mode function, an optional parameter for either rounding or binning floats. The mean and median functions will work fine with floats in their raw form, but thereās a decent chance that an analog signal recorded to a float will have enough unique values over a short period to have all of the values be unique. Including the rounding/binning actions in the mode function should keep from having to loop through the data twice.
When I originally created this thread I was thinking in Neural Network and Machine Learning kind of functions and capabilities. It is nice to see it evolving into something as useful and needed. What Iām thinking is a github with all these kind of basic and necessay math functions. So far is empty but Iāll be feeding it at some point in the near future.
If someone wants to participate or collaborate (so we can have a more robust library) feel free to say so and then I can add you as collaborators.
P.S. Iām fairly new to github (as you can see in the git profile, so feel free also to add structrue if needed)
Hi everyone,
Iāll throw my $0.02 in as well. Iām looking to do nonlinear regression on data that I am capturing with Ignition. The expectation function is y = A*(x + B)^(-C) + D so I fear it is not as easy as using Jordanās polynomial regression function (tantalizingly close though).
Currently I have to capture the dataset in Ignition, export it into Minitab where I do the regression, and then enter my A,B,C, and D coefficients back into Ignition where I take the derivatives of the function. Iād like to avoid the need for Minitab.
Wouldāve been easy in scipy, but we all know thatās not an option. I canāt decipher if either the JScience or Apache Math collections can do this kind of regression. Can they? If so, how do I implement them?
If not, Jordan, what are my chances of you adding this kind of number crunching to your library??! Do you like donuts??
Cas