I'm currently working with dataset and I'm a bit lost on how to do it efficiently. So I have some questions.
Whenever I do ds = system.dataset.setValue(ds, 0, "COLUMN_1", "test") what happens to the original ds? It gets lost in memory? or maybe it gets deleted?
Whenever I try to modify or do operations with dataset entire columns I do something like getColumnAsList() > map(f, list) > addColumn() with same name. Wouldn't make more sense to simply have a list of lists?
And last, whenever I call a query or I'm workings with dataset should I delete or drop dataset? Somethink like when you are working in C or C++.
Datasets in Ignition are Immutable. All of the system.dataset.* functions that return a dataset, return a new dataset with the modification applied. I suppose you could say that the old dataset is deleted, but I'm not certain that is fully accurate under the hood. In reality, there isn't any reason to worry about what happens to the old dataset. In other words things like releasing memory are all handled for you.
I'm not sure why you would need to loop through all of the columns of a dataset to add a column, I also don't know what you're trying to state with the map(f, list). In other words, I'm not sure what modification you're attempting to indicate with your pseudo code.
You should probably investigate PyDatasets which allow you to work with datasets like python lists and dictionaries.
It is extremely common to use a python list of lists (list of rows) as an intermediate value when working with datasets. You would loop through a source dataset (immutable) build a new list of rows, and then convert to dataset at the end with system.dataset.toDataSet(). Or, if you need fine control over column types, use the DatasetBuilder helper class. (Lots of examples of both techniques here on the forum.)