Python List Comprehensions Help

Ignition
81

I have a curveball.

When I want to write the subset of a list that exists to another new list

make a new list of only the subsets to write from?
Or is there an easy way to reference and exclude parts of a list that exists?

I have 68 values, and only need to write 51 of them

82

Depends, are the values contiguous or not?

If you have a list and you only want to use some contiguous part of them:

print thisList[startIndexOfGroup:endIndexOfGroup]

If not you’re better off just making a new list of only the subset you need.

Also, remember that depending on where you are using it, sometimes a generator statement is acceptable so you can just use a comprehension to “select” the parts of the list you need.

sum([i for i in range(1,101,10)])

is functionally equivalent to:

sum(i for i in range(1,101,10))

1 Like
83

You’re most likely gonna create a new list, that’s what a comprehension does anyway.
As @lrose said, you can ‘slice’ a list (and other things, strings for example), and it’s quite flexible.

s = "foo bar"
s[:3] == "foo"
s[:-4] == "foo"
s[4:] == "bar"
s[-3:] == "bar"
s[::2] == "fobr"
s[1::2] == "o a"
s[::-1] == "rab oof"

[start:end:step], negative numbers start from the end, and a negative step, well, steps backwards.

If the subset of the list doesn’t follow a clear pattern, then you’ll have to filter it depending on whatever it is you want from each element. You could use list.filter(), but comprehensions are the simplest way of doing this:

new_list = [element for element in old_list if pick_this(element)]

you could even mix and match everything, picking elements from a slice in a comprehension, using complex conditions, etc…

new_list = [element for i, element in enumerate(old_list[1:-3::-2]) if select_this(element) and sqr(i) not in [2, 4, 6]]
1 Like
84

I read about slices a few months ago, and forgot.

Thanks so much for these examples. They are really helpful, I think more helpful than the resource I read from previously.

Update: the giant script is revised!
14 pages to 2 pages
so many fewer writeBlocking and readBlocking

thanks you very much
I think I can revise all the other scripts easy now

I need to save up to get a couple rounds if we have an Ignition convention

85

Do you have a preferred way to use the script console to show you the value of a historized tag at a certain time?

Seems like something handy to use frequently for testing these scripts.

86

https://docs.inductiveautomation.com/display/DOC81/system.tag.queryTagHistory

1 Like
87

What do you mean ?
Some comprehensions and data manipulation tricks to use on the results of queryTagHistory ?
queryTagCalculations usually does enough for me not to have to hack my way through the data, though.

Aside from that, you’ll need to be more specific about what you’re looking for.

88

Thanks

I went to the Ad Hoc Trends that I got from the exchanged and tried to pull up a categorical tag value

Did not work

Was trying to remember system.tag.queryTagHistory thanks so much

I don’t know how to view the dataset.
print dataSet

output
Dataset [1R ⅹ 2C]

tried print dataSet.value, dataSet [0], dataSet[0,0]

oh found this Datasets - Ignition User Manual 8.1 - Ignition Documentation

89

You can convert to a py dataset, which you may find easier to work with.

1 Like
90

You’ll need to iterate through it.
But I hate datasets and usually convert them to pydatasets:

tags = system.tag.queryTagHistory(tag_paths)
tags = system.dataset.toPyDataSet(tags)
for tag in tags:
	tag
1 Like
91 
endTime = system.date.now()
startTime = system.date.addMinutes(endTime, -30)
dataSet = system.tag.queryTagHistory(paths=['[test]tag'], startDate=startTime, endDate=endTime, returnSize=1, aggregationMode="LastValue", returnFormat='Wide')
this = system.dataset.toPyDataSet(dataSet)

for row in this:
	for value in row:
		 print value

output

Wed Mar 16 09:37:34 EDT 2022
92

See my example.

Also, if endTime is now, you can instead use rangeMinutes or rangeHours:

system.tag.queryTagHistory(paths, rangeHours=48)
93

Thanks
looks like the scripts were not the culprits.
I think at least in 8.1.0 if you pass a noneType to the aliases in the report, it blows up the datasource that sets those.

image

so using this technique

dataSet = system.tag.queryTagHistory(paths=['[Test]tag'], startDate=startTime, endDate=endTime,   returnFormat='Wide')
this = system.dataset.toPyDataSet(dataSet)
for row in this:
	for value in row:
		 print value
		 
print ('standard vol 2')

or teh one you showed me

tags = system.tag.queryTagHistory(tag_paths)
tags = system.dataset.toPyDataSet(tags)
for tag in tags:
	tag

I was able to figure it out

I think my dropdown had like “select” as an option and an X to cancel the value.
So I was passing nonetypes when I had this showing on the dropdown and passing that value as a parameter to be my report’s data source for alias keys

not really list comprehensions, but I asked about the script to troubleshoot them and got the results.

used

endTime =  system.date.addHours(system.date.now(),-10)
startTime = system.date.addHours(system.date.now(), -24)
94

Today, I solo made a list comprehension.
Though maybe it is nearly redundant with thread.

This = [paths[i] for i in range(0,15) if line = machineIDs[i]]

Supposing they are all defined and paths is as long as machineIDs, 14 or greater, then this will get the the path based on the machineID.

Edit: The script doesn’t error, it returns:
[]
So I am not sure what to do.

I was going to write a variable to the path returned.
Maybe I write like

if this != [] :
   system.tag.writeBlocking(this)
95

if line = machineIDs[i] uses an assignment instead of a comparison. Should be:

lines = [paths[i] for i in range(15) if line == mahcineIDs[i]]

Also, you will need values to use in your write so:

if lines:
    system.tag.writeBlocking(lines,values)
1 Like
96

Stop calling things this :X

[path for path, machine in zip(paths, machineIDs) if line == machine]

This also supposes paths and machineIDs are the same length.
If they’re not, it can be mitigated by using izip_longest from itertools:

from itertools import izip_longest as zipl
[path for path, machine in zipl(paths, machineIDs) if line == machine]

Now, I’m expecting the machines Ids to be unique (machineIDs could be a set).
Which means the resulting list will always either be empty (machine not found) or contain only one element.
If that’s the case, instead of using a list comprehension, which will examine every element, you can use next() and pass it a generator expression. Which is basically a list comprehension without the brackets:

from itertools import izip_longest as zipl
this = next(path for path, machine in zipl(paths, machineIDs) if line == machine)

this will NOT be a list, but be the value of path for wich line matched the machine id.
If no match is found, it raises a stopIteration. Which you can use in a try/except:

from itertools import izip_longest as zipl
try:
    this = next(path for path, machine in zipl(paths, machineIDs) if line == machine)
except StopIteration:
    logger.warn("no matching machine id found")

OR, you can pass next() a second parameter, that it will return if no match is found.

from itertools import izip_longest as zipl
this = next((path for path, machine in zipl(paths, machineIDs) if line == machine), None)
if this is None:
    logger.warn("no matching machine id found")

Note the added parentheses. That’s because with just one parameter, next() figures out that it’s a generator expression. With another parameter, you need to help it out a bit.

3 Likes
97

How do I know which to choose between?

speedPath =[path for path, machine in zip(paths, machineIDs) if line == machine]

and

speedPath = [paths[i] for i in range(15) if line == machineIDs[i]]

What you had just posted is a little difficult for me to grasp. I get some of it.

98

They basically do the same thing, but the first one will work regardless of the length of your iterables (as long as both are the same length).
The second one will only work if they contain 16 elements. It’s also less pythonic.

I’m guessing generator expressions might be a source of confusion. They’re largely documented on the internet, but the gist of it is that they generate items. Iterating through them will produce items one at a time.
They’re the basis of comprehensions, which are nothing more than a generator expression enclosed in brackets, squared for lists, curly for dicts.
example:

things_squared = [item**2 for item in things]
# is equivalent to
things_squared = list(item**2 for item in things)

The brackets are just sugar to make a list out of a genexpr.

Now, why do I suggest using a genexpr and next ?
Using a list comprehension will go through the whole iterable:

squares = [i**2 for i in range(10)]

Will make the whole list, as it should.
But if you’re only interested in a specific element, ie:

nine = [i**2 for i in range(10) if i == 3][0]

This will also compute the whole list, even though you’re only interested in the 4rth element.
Whereas

nine = next(i**2 for i in range(10) if i == 3)

Will iterate through the items returned by the genexpr until a match is found, then it will stop.
While the difference is not significant on a 16 items list, it could make a huge difference with big lists.
Also, it’s slightly prettier and more pythonic.
It also offers a different mechanic in the case no match is found.
With a list comprehension, you’ll have to make sure the list is not empty before you can index it:

hundred = [i**2 for i in range(5) if i == 10]
if hundred:
    print("hundred  = {}".format(hundred[0]))
else:
    print("there's no hundred here")

# or 
try:
    hundred = [i**2 for i in range(5) if i == 10][0]
except IndexError:
    print("there's no hundred here")
else:
    print("hundred = {}".format(hundred))

The default return parameter that you can pass to next can make things quite cleaner.

2 Likes
99

range(15) is hard-coded, so it is less flexible if that number changes. If you use zip(paths, machineIDs) then your script is more dynamic since it can handle it no matter what the length of machineIDs and paths is.

Note what @pascal.fragnoud said, that zip only works if the length of paths = the length of machineIDs, otherwise use from itertools import izip_longest as zipl

1 Like
100

Well, zip will "work" with iterables of different length, it will just stop when one of them is finished. Which might be the behavior he's looking for (It actually makes sense to stop looking for matches after the end of one of the iterables)
Where doing it with range(15), if either one of paths or machineIDs is shorted than 16 elements, will result in a IndexError being raised.

3 Likes