# Script help: Holiday edge case lead to empty datasets

**URL:** <https://forum.inductiveautomation.com/t/script-help-holiday-edge-case-lead-to-empty-datasets/66975>\
**Category:** Ignition\
**Created:** [November 28, 2022, 3:07pm UTC](https://forum.inductiveautomation.com/t/script-help-holiday-edge-case-lead-to-empty-datasets/66975 "2022-11-28T15:07:53Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![zacharyw.larson](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/zacharyw.larson/32/23749_2.png) [@zacharyw.larson](https://forum.inductiveautomation.com/u/zacharyw.larson)\
**Post date:** [November 28, 2022, 3:07pm UTC](https://forum.inductiveautomation.com/t/script-help-holiday-edge-case-lead-to-empty-datasets/66975/1 "2022-11-28T15:07:53Z")

</div>

```auto
for row, nextRow in zip(datasetIn, datasetIn[1:]):

```

After a long holiday, I don't have any data.  
So when I reference nextRow, I get an error that I tried to call an undefined nextRow.

So to fix this, I thought maybe:

```auto
try:
     hoursSum= nextRow['Hours']
     hoursSum=0
except:
      return value

```

However, I am still getting an error that farther down, the script is erroring.

* * *

I figured it out sort of.  
Looks like I needed the try error outside of the for loops because I don't think the for loop runs when there are no rows.

```auto
for row, nextRow in zip(datasetIn, datasetIn[1:]):
   #Location 1
   try:
       hoursSum= nextRow['Hours']
    hoursSum=0
    except:
        return value
    #chunk of code here manipulating row

#Location 2
 try:
    hoursSum= nextRow['Hours']
    hoursSum=0
except:
     return value
#chunk of code here manipulating nextRow ending with a return of a new dataset

```

---

<div class="post-metadata">

**Author:** ![jlandwerlen](https://avatars.discourse-cdn.com/v4/letter/j/bc8723/32.png) [@jlandwerlen](https://forum.inductiveautomation.com/u/jlandwerlen)\
**Post date:** [November 28, 2022, 3:13pm UTC](https://forum.inductiveautomation.com/t/script-help-holiday-edge-case-lead-to-empty-datasets/66975/2 "2022-11-28T15:13:21Z")

</div>

> [@zacharyw.larson](#):
>
> After a long holiday, I don't have any data.

I would fix that problem first 🙂

---

<div class="post-metadata">

**Author:** ![pascal.fragnoud](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pascal.fragnoud/32/24728_2.png) [@pascal.fragnoud](https://forum.inductiveautomation.com/u/pascal.fragnoud)\
**Post date:** [November 28, 2022, 3:14pm UTC](https://forum.inductiveautomation.com/t/script-help-holiday-edge-case-lead-to-empty-datasets/66975/3 "2022-11-28T15:14:34Z")

</div>

[https://docs.python.org/2/library/itertools.html#itertools.izip\_longest](https://docs.python.org/2/library/itertools.html#itertools.izip_longest)

```python
from itertools import izip_longest as zipl

foo = [1]
print [(row, next_row) for row, next_row in zipl(foo, foo[1:])]
foo = []
print [(row, next_row) for row, next_row in zipl(foo, foo[1:])]

```

> `[(1, None)]`  
> `[]`

---

<div class="post-metadata">

**Author:** ![zacharyw.larson](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/zacharyw.larson/32/23749_2.png) [@zacharyw.larson](https://forum.inductiveautomation.com/u/zacharyw.larson)\
**Post date:** [November 28, 2022, 3:24pm UTC](https://forum.inductiveautomation.com/t/script-help-holiday-edge-case-lead-to-empty-datasets/66975/4 "2022-11-28T15:24:07Z")

</div>

@jlandwerlen  
There is nothing to "fix" about not having data over a long holiday.  
If nothing runs for four days, there is no data for the last 3 days.

@pascal.fragnoud  
That code from the python site is using so many things I don't know that I don't understand it's details about izip\_longest.  
I am not sure what you are saying.

You mean the for loop is running, it returns an empty list when empty?

---

<div class="post-metadata">

**Author:** ![jlandwerlen](https://avatars.discourse-cdn.com/v4/letter/j/bc8723/32.png) [@jlandwerlen](https://forum.inductiveautomation.com/u/jlandwerlen)\
**Post date:** [November 28, 2022, 3:29pm UTC](https://forum.inductiveautomation.com/t/script-help-holiday-edge-case-lead-to-empty-datasets/66975/5 "2022-11-28T15:29:59Z")

</div>

> [@zacharyw.larson](#):
>
> There is nothing to "fix" about not having data over a long holiday.  
> If nothing runs for four days, there is no data for the last 3 days.

Good point, so your code needs to handle not having data.

---

<div class="post-metadata">

**Author:** ![pascal.fragnoud](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pascal.fragnoud/32/24728_2.png) [@pascal.fragnoud](https://forum.inductiveautomation.com/u/pascal.fragnoud)\
**Post date:** [November 28, 2022, 3:34pm UTC](https://forum.inductiveautomation.com/t/script-help-holiday-edge-case-lead-to-empty-datasets/66975/6 "2022-11-28T15:34:15Z")

</div>

The code from the website is just here to show a possible implementation of `izip_longest`. It's supposed to help understand what the function does.

I'm not sure what your use case is, as the code you showed makes little to no sense to me, so this may not be pertinent to your issue.  
If I knew what you're trying to do, I could maybe be more helpful, but right now I don't really see the issue you ran into...

I mean, you say you get an issue about an undefined nextRow, but I have no idea where and how you're calling this variable. The only way using a variable defined in this way (ie: `x` defined with `for x in something`) is calling it outside of the loop. Which clearly, you shouldn't do.  
Your edit does show a little more details, but the indentation is messed up and there's no way this can ever run.  
You're also nullifying an assignment (`hoursSum = nextRow['hours']`) by immediately afterward setting it to 0.

---

<div class="post-metadata">

**Author:** ![zacharyw.larson](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/zacharyw.larson/32/23749_2.png) [@zacharyw.larson](https://forum.inductiveautomation.com/u/zacharyw.larson)\
**Post date:** [November 28, 2022, 3:42pm UTC](https://forum.inductiveautomation.com/t/script-help-holiday-edge-case-lead-to-empty-datasets/66975/7 "2022-11-28T15:42:57Z")

</div>

I edited the top post as I figured a way to catch the error.  
Thanks for helping me and showing me that doc page.

I figured the script purpose didn't matter much.  
I think "an error of calling a variable before it is defined" might be so common that there is a common solution.

---

<div class="post-metadata">

**Author:** ![pascal.fragnoud](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pascal.fragnoud/32/24728_2.png) [@pascal.fragnoud](https://forum.inductiveautomation.com/u/pascal.fragnoud)\
**Post date:** [November 28, 2022, 3:48pm UTC](https://forum.inductiveautomation.com/t/script-help-holiday-edge-case-lead-to-empty-datasets/66975/8 "2022-11-28T15:48:58Z")

</div>

> [@zacharyw.larson](#):
>
> I think "an error of calling a variable before it is defined" might be so common that there is a common solution.

There is a common solution: Don't do it.  
Now, how not to do it depends on how it was done. Which is why seeing your code would be helpful.  
I feel a try/catch is probably not the best solution to this problem.

---

<div class="post-metadata">

**Author:** ![lrose](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/lrose/32/62724_2.png) [@lrose](https://forum.inductiveautomation.com/u/lrose)\
**Post date:** [November 28, 2022, 3:49pm UTC](https://forum.inductiveautomation.com/t/script-help-holiday-edge-case-lead-to-empty-datasets/66975/9 "2022-11-28T15:49:32Z")

</div>

> [@zacharyw.larson](#):
>
> I think "an error of calling a variable before it is defined" might be so common that there is a common solution.

There is, don't do it.

The problem here is that you are trying to index an empty list. You should make sure that there is data in the list prior to trying to access it if there is a chance that the list is empty.

```python
#assuming that datasetIn is a pyDataSet
if datasetIn.rowCount:
    for row, nextRow in zip(datasetIn, datasetIn[1:]):

```

---

<div class="post-metadata">

**Author:** ![zacharyw.larson](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/zacharyw.larson/32/23749_2.png) [@zacharyw.larson](https://forum.inductiveautomation.com/u/zacharyw.larson)\
**Post date:** [November 28, 2022, 3:52pm UTC](https://forum.inductiveautomation.com/t/script-help-holiday-edge-case-lead-to-empty-datasets/66975/10 "2022-11-28T15:52:33Z")

</div>

Thanks

If I want to avoid running my code when 0 or 1 row, this is a good technique?

`if datasetIn.rowCount>1:`

---

<div class="post-metadata">

**Author:** ![pascal.fragnoud](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pascal.fragnoud/32/24728_2.png) [@pascal.fragnoud](https://forum.inductiveautomation.com/u/pascal.fragnoud)\
**Post date:** [November 28, 2022, 3:54pm UTC](https://forum.inductiveautomation.com/t/script-help-holiday-edge-case-lead-to-empty-datasets/66975/11 "2022-11-28T15:54:47Z")

</div>

If what you want is not to run something on a dataset that has less than 2 rows, then yes.  
But if there's a reason why you don't want to run something on this dataset, then...

> [@pascal.fragnoud](#):
>
> Now, how not to do it depends on how it was done. Which is why seeing your code would be helpful.

---

<div class="post-metadata">

**Author:** ![zacharyw.larson](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/zacharyw.larson/32/23749_2.png) [@zacharyw.larson](https://forum.inductiveautomation.com/u/zacharyw.larson)\
**Post date:** [November 28, 2022, 4:30pm UTC](https://forum.inductiveautomation.com/t/script-help-holiday-edge-case-lead-to-empty-datasets/66975/12 "2022-11-28T16:30:01Z")

</div>

~~On that doc page, that is the function definition for that library?  
I thought it was an example, but it was so complicated. ~~

It must be a complicated example.

```auto
class ZipExhausted(Exception):
    pass

def izip_longest(*args, **kwds):
    # izip_longest('ABCD', 'xy', fillvalue='-') --> Ax By C- D-
    fillvalue = kwds.get('fillvalue')
    counter = [len(args) - 1]
    def sentinel():
        if not counter[0]:
            raise ZipExhausted
        counter[0] -= 1
        yield fillvalue
    fillers = repeat(fillvalue)
    iterators = [chain(it, sentinel(), fillers) for it in args]
    try:
        while iterators:
            yield tuple(map(next, iterators))
    except ZipExhausted:
        pass

```

I don't know:  
ZipExhausted  
Raise  
Yield  
Repeat  
Chain

I will look them up.

---

<div class="post-metadata">

**Author:** ![lrose](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/lrose/32/62724_2.png) [@lrose](https://forum.inductiveautomation.com/u/lrose)\
**Post date:** [November 28, 2022, 5:00pm UTC](https://forum.inductiveautomation.com/t/script-help-holiday-edge-case-lead-to-empty-datasets/66975/13 "2022-11-28T17:00:50Z")

</div>

The `zip()` function by itself will truncate a longer list to the length of the shortest.

So

```python
list1 = ['A','B','C','D']
list2 = ['x','y']

for i,j in zip(list1,list2):
    print i,j

```

Will yield

```auto
Ax
By

```

Where this:

```auto
from itertools import izip_longest as zipl

list1 = ['A','B','C','D']
list2 = ['x','y']

for i,j in zipl(list1,list2):
    print i,j

```

Will yield:

```auto
Ax
By
C
D

```

`raise` will "raise" an exception. In this case they have defined a default exception `ZipExhausted`.

`yield` is used to "return" the next iteration in a generator.

`repeat` and `chain` are other functions inside of the `itertools` package.

---

<div class="post-metadata">

**Author:** ![bkarabinchak.psi](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/bkarabinchak.psi/32/83567_2.png) [@bkarabinchak.psi](https://forum.inductiveautomation.com/u/bkarabinchak.psi)\
**Post date:** [November 28, 2022, 5:21pm UTC](https://forum.inductiveautomation.com/t/script-help-holiday-edge-case-lead-to-empty-datasets/66975/14 "2022-11-28T17:21:16Z")

</div>

I feel like this has become overly complicated.

I don't see your full code but it seems like you need to look at your current data and your next row's data to do whatever it is you need to do. Is that correct?

I don't really use zip I am sure there is a way to with it but it would confuse my coworkers so here's how I would do it with enumerate -

```auto
l = [1,2,3,4]

for row, item in enumerate(l):
	try:
		curItem = item
		nextItem = l[row+1]
        # Do something with both row's data
	except IndexError:
		# You are at the last item in list so you can stop now
		# pass
        break

```

Is there some other condition you have that makes this method unsuitable for you?

---

<div class="post-metadata">

**Author:** ![lrose](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/lrose/32/62724_2.png) [@lrose](https://forum.inductiveautomation.com/u/lrose)\
**Post date:** [November 28, 2022, 5:26pm UTC](https://forum.inductiveautomation.com/t/script-help-holiday-edge-case-lead-to-empty-datasets/66975/15 "2022-11-28T17:26:59Z")

</div>

> [@bkarabinchak.psi](#):
>
> Is there some other condition you have that makes this method unsuitable for you?

The real issue of the thread was trying to index an empty list.

> [@Script help: Holiday edge case lead to empty datasets](https://forum.inductiveautomation.com/t/script-help-holiday-edge-case-lead-to-empty-datasets/66975/9):
>
> There is, don't do it. The problem here is that you are trying to index an empty list. You should make sure that there is data in the list prior to trying to access it if there is a chance that the list is empty. #assuming that datasetIn is a pyDataSet if datasetIn.rowCount: for row, nextRow in zip(datasetIn, datasetIn[1:]):

The rest is just in response to @pascal.fragnoud recommending the `izip_longest` function and the example code using some built-ins and functions without much context to why they're used.

`zip()` works just fine for what he is trying to accomplish, so long as there is actually data in the dataset.

---

<div class="post-metadata">

**Author:** ![bkarabinchak.psi](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/bkarabinchak.psi/32/83567_2.png) [@bkarabinchak.psi](https://forum.inductiveautomation.com/u/bkarabinchak.psi)\
**Post date:** [November 28, 2022, 5:35pm UTC](https://forum.inductiveautomation.com/t/script-help-holiday-edge-case-lead-to-empty-datasets/66975/16 "2022-11-28T17:35:07Z")

</div>

Oh I see so sometimes datasetIn was zero rows, so `zip(datasetIn, datasetIn[1:])` was causing the error due to `datasetIn[1:]` throws an IndexError. I see the that `zip(datasetIn, datasetIn[1:])` makes sense to me now, pretty nifty.

Yea seems the easiest way would just be checking dataset length prior and only running it if there's enough data present or doing something else if there is not.

@zacharyw.larson I would not do a deep dive into itertools and all the other python generator keywords, I would just do your `if datasetIn.rowCount>1:` check prior to running your script.

---

<div class="post-metadata">

**Author:** ![pturmel](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pturmel/32/3295_2.png) [@pturmel](https://forum.inductiveautomation.com/u/pturmel)\
**Post date:** [November 28, 2022, 5:50pm UTC](https://forum.inductiveautomation.com/t/script-help-holiday-edge-case-lead-to-empty-datasets/66975/17 "2022-11-28T17:50:03Z")

</div>

The technique suggests the source SQL needs a LEAD() function for columns of interest. Which will automatically deliver a null in the last row, and greatly simplify the jython. (The DB will almost certainly be faster, too.)

---

<div class="post-metadata">

**Author:** ![zacharyw.larson](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/zacharyw.larson/32/23749_2.png) [@zacharyw.larson](https://forum.inductiveautomation.com/u/zacharyw.larson)\
**Post date:** [November 28, 2022, 6:04pm UTC](https://forum.inductiveautomation.com/t/script-help-holiday-edge-case-lead-to-empty-datasets/66975/18 "2022-11-28T18:04:11Z")

</div>

[I liked learning about the keywords on the w3schools.](https://www.w3schools.com/python/python_ref_keywords.asp)

I hadn't realized there were so many more that I didn't know about.

---

<div class="post-metadata">

**Author:** ![pascal.fragnoud](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pascal.fragnoud/32/24728_2.png) [@pascal.fragnoud](https://forum.inductiveautomation.com/u/pascal.fragnoud)\
**Post date:** [November 28, 2022, 7:18pm UTC](https://forum.inductiveautomation.com/t/script-help-holiday-edge-case-lead-to-empty-datasets/66975/19 "2022-11-28T19:18:44Z")

</div>

The function is not an example of its use, it's what its implementation might look like - though it's probably much more complex than that - to give you a better understanding of what it does.

But I suggested that function because I misunderstood your problem, you probably don't need it in this case.

---

<div class="post-metadata">

**Author:** ![pascal.fragnoud](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pascal.fragnoud/32/24728_2.png) [@pascal.fragnoud](https://forum.inductiveautomation.com/u/pascal.fragnoud)\
**Post date:** [November 28, 2022, 7:26pm UTC](https://forum.inductiveautomation.com/t/script-help-holiday-edge-case-lead-to-empty-datasets/66975/20 "2022-11-28T19:26:19Z")

</div>

> [@bkarabinchak.psi](#):
>
> causing the error due to `datasetIn[1:]` throws an IndexError.

'Slicing' an empty list should not raise errors. It just returns an empty list.

Which is used in some... not so pretty technics:

```python
some_list = ["foo", "bar", "baz"]
x = [some_list[1:2] or ['not found']][0]
# x == "bar"

some_list = []
x = [some_list[1:2] or ["not found"]][0]
# x == "not found"

```

and things like that. Don't do it :X  
It's simpler and clearer to just check if `len(some_list) > the_index_you_want_to_use`

The error was/is probably caused by an use of the variable declared in the loop definition outside of the loop itself, something like:

```python
for x in []:
    pass
print x

```

[Next page](https://forum.inductiveautomation.com/t/script-help-holiday-edge-case-lead-to-empty-datasets/66975.md?page=2)