[Feature] dataset with an indexed column and search function

Ignition Early Access
1

Dataset are everywhere in Ignition.It’s very handy but sometimes we need to search a specific value in a column.
A full scan of a large dataset can be time consuming.
That why if we can define that one column is indexed and have a search function script based on that index.

2 Likes
2

I’m not sure what this would look like. Is this similar to this other post: http://forum.inductiveautomation.com/t/feature-query-ignition-dataset/20348/5 , where you want a SQL-like ability to access the dataset? I’m not sure how an “index” column would otherwise help in looking at a large dataset.

3

SQL-like would be good, but SQL Query without index is not better than a loop to full-scan all the dataset.
I think about script function like :

  • system.dataset.addIndex(numColumn)
  • system.dataset.find(numColumn,keyValue) => return the row
4

Have you used the .getColumnAsList() method? With a list, jython’s .index() method will give you the position (row number) of a value in that column. To go faster, load that column into a dictionary’s keys, with the values being the row number. Voilà, an index. The jython code that does these searches is as optimized as can be.

1 Like
5

Yes indeed, that do the job. With the need to update the dictionnary each time you add/remove rows in the dataset. A new user type of dataset with these dictionnary update done in the background can be done, but it’s better if it was built-in in Ignition script system… there are lots of function to manipulate dataset, but nothing to seach in a dataset…
That’s why in script we ofen prefer transfert data in dictionnary ou list to manipulate the data…and it add lots of plumbing code…

6

Well, that's a new dataset anyways, so it'd have to be a new index. I'll see about adding a searchable dataset implementation to my Simulation Aids module.

1 Like
7

Phil,

Do you know where the .getColumnAsList() method is documented? I searched the user manual and couldn’t find a reference. Just curious how you discovered this function. Was very useful by the way!

8

It is a method of the BasicDataset class (Dataset is an Interface). BasicDataset is the root of pretty much all of the dataset classes that you’ll encounter in Vision.