I am using dataset manipulation for some graphs and running into some limitations with the current libraries I have available to me, unless I’m missing a tool that’s already available. Some examples include concatenating multiple datasets together as well as updating an entire column with a list. I’m not sure if the update column option is available for a dataset anyways but it would be nice to have just to simplify some code.
I looked into Pandas some and it seemed like good options. My question would be what are some other’s experience in using this libraries, and/or others, and what would be some things to watch out for, just so I don’t crash the gateway when importing or using certain functions.
Also any suggestions for some other useful libraries would be welcome as well.
Edit:
The question changed through some of the comments so ignore the first two paragraphs. I have an idea on how I am going to do this by just combining the original datasets into one but I am still curious how to create a union two or more similar datasets with only the rows being different and essentially just stacking them on top of each other?
for row in range(dataset.rowCount):
if not len(newValues) == dataset.rowCount:
break
dataset = system.dataset.setValue(dataset,row,'yourColumnName',newValues[row])
I haven’t run into anything I’ve needed to do with a dataset that I cant do with the system.dataset.* functions.
Perhaps I’m mis-understanding what it is that you’re looking for?
Yea I have 5 different dataset tags with the tag paths and names for different trending data that I want to see. What I am trying to do is to have the ability to compare and contrast multiple of these datasets on the same graph, for example, to take temperature and feed trends and see if there is any relation, or really anywhere where one factor inside the barns may affect something else. This also needs to be able to filter each of the datasets but I already am able to do that with just taking an unfiltered dataset and dropping the rows that I don’t want when applying the filter.
So system.dataset.appendDataset() allows you to take 2 similar datasets and essentially add all rows off of dataset 2 to dataset 1?
I elaborated on my use case in my reply to @lrose. Would using the database through queries be a better/more efficient way of doing what I am trying to do? I just don’t a whole lot of experience using the databases in general so I’m not super effective in utilizing them.
If you want to show 5 different datasets on a trend, then use a standard chart component. Add 4 datasets to it, using the chart customizer, and then just bind the datasets to your tags.
Or, assuming that they have the same number of rows, you could add multiple columns to a single dataset and then set the charts data equal to that dataset in a script. Something like this:
#note I am assuming that your dataset tags have the same structure where the value column is the same for each
tagPaths = ['path/to/dataset1','path/to/dataset2']
tagDataSets = [tag.value for tag in system.tag.readBlocking(tagPaths)]
baseDataset = tagDataSets[0]
for tagDS in range(1,len(tagDatasets)):
baseDataset = system.dataset.addColumn(baseDataset,[tagDS.getValueAt(row,'valueColumn') for row in range(tagDS.rowCount)],tagPaths[tagDs],float)
Honestly though, this sounds like you’re doing something that will eventually come back to haunt you. Are you trying to store “historical” data in a Dataset Tag? How big do you expect these dataset tags to get?
I had created an easy chart with each of the different charts that I wanted, then took the Tag Pens dataset and copied it to a dataset memory tag. I then filtered the data and wrote the new Tag Pens data to the easy chart Tag Pens property. So none of the data is being stored in the dataset tag, all that is is a memory tag that stores the tag pens for my easy chart. I should try the standard chart component out though, maybe I’ll figure something out with it. There are about 530 tags that I am using in this which are organized into the different memory tags.
You can enable and disable tags in the Tag Pens dataset, so there isn’t a need for multiple charts, and I’m not certain why you want to copy the Tag Pens data out?
When you say there are 530 tags, do you mean there are 530 different chart configurations?
Yea that’s kind of like what I am trying to create except I need it to be where if I deselect Barn 708 and Barn 601, then it won’t show those barns. Also when I “add” average temperature and average feed, for it to self populate with all of the tags that I want it to while also filtering out the barns that I don’t want it to show, if all barns are selected then a total of 10 tags are put onto the trend for each “selection” I make. The reason for wanting to copy the Tag Pens data out is to try and simplify the code so all of the data isn’t a conglomeration on the same dataset. Plus with the way I am doing this currently with the datasets none of it is enabling or disabling the different tags, I just drop the rows off of the dataset that are unwanted and set the Tag Pens to the new dataset.
What I meant by 530 different tags is that I have 530 different numerical values that are trending, 53 tags per barn. For example there are 16 temperature probes per barn plus an average temperature. Same thing goes with CFM.
I know I can do this simply by combining the datasets* and messing around with my scripting so that it finds the correct values and utilizes them but I would prefer it if I could keep them separated if it isn’t too difficult to do so.
To try and simplify my goal is to make an “Ad-Hoc” chart that is able to seamlessly add and remove many different tags without having to select and deselect 100 different tags. Essentially adding and removing tag groups rather than individual tags.
Edit:
I’m not sure if there is truly a good reason to keep those dataset’s separate to be honest like I had mentioned in paragraph 3. My original reasoning was to simplify the code but realistically that is up to how I type it up as well as how well the data is organized.
You weren’t misunderstanding what I was looking for I was just confusing myself by not reading up on system.dataset.appendDataset() and thinking that the solution was more complicated than it actually is, that was exactly what I was needing. Thank you for your help.