Hello all,
I'm at a roadblock implementing a requested feature for an alarm pipeline.
When an alarm reaches a notification block, I want it to begin calling users on a on-call roster, BUT if a user higher on the priority list is called and if they acknowledge the alarm I want it to discard the rest of the roster and move on to the subsequent blocks down the pipeline chain.
I currently have a workaround but it isn't compatible with alarm consolidation - (here is a diagram of what I have so far as well as an explanation of why it isn't working):
I also have a calculated roster in my notification block:
"""
Function's purpose is to dynamically pull from roster user by user/one by one to
trickle down the complete/whole roster and evaluate user by user if someone
has ack'ed the alarm. This prevents subsequent operator calls if someone higher
on the on-call roster has ack'ed the alarm.
Uses 2 local pipeline properties: rosterCounter(integer) and atRosterLimit(boolean)
to determine whether to exit out of this loop and continue down the pipeline.
"""
# get roster of interest
roster = system.alarm.getRosters()["WaterOnCall"]
rosterSize = len(roster)
# index
counter = event["rosterCounter"]
"""
evaluate if cycle is at limit
if at limit set atRosterLimit flag to true
still call out because there is one user left
subsequent logic will check if ack'd else move to second stage alert
"""
if rosterSize == counter + 1:
l.info("Reached roster limit...Setting atRosterLimit flag to true")
event["atRosterLimit"] = 1
### * IN TEST * ###
"""
Custom roster build loop issue:
If consolidation happens memory space is shared and consolidated together as well
as their local variables for a local time
The first alarm to become true inherits all alarms within the consolidation time window,
once these alarms move on from their "neighborhood scope" they split off and become
individual entities again
While the inheritor alarm and inherited alarm(s) are in sync, each alarm still gets its own set of
variables. This can be seen after it exits this custom loop because the counter begins to go
out of bounds after.
"""
# get user data
user = system.user.getUser("default", roster[counter])
# get contact data
contactObj = user.getContactInfo()
# build and return single user roster for callout op
number = None
for i in user.getContactInfo():
if i.getContactType() == "phone":
number = i.value
builder.username(roster[counter]).phone(number).add()
uList = builder.build()
return uList
- I begin this loop by setting 2 variables:
- rosterCounter: This variable keeps track of what user in the on-call roster we want to call
- atRosterLimit: This flag is set to true if the iteration sees that we are at the end of the roster
- The calculated roster reads in the rosterCounter index and uses that to access which user to call
- The adjacent expression block checks if the user has acknowledged the alarm and/or if we are at the end of the roster
- If the user has acknowledged the alarm, move on to the next sequence in the alarm pipeline
- Else, increment the rosterCounter index and attempt to call the next user
Issue
For single alarms, this solution works great. But when consolidation happens for multiple alarms, through testing I can see that the first alarm becomes what you could call the "inheritor".
The group of alarms become apart of the first alarm to go active.
It seems that the memory space becomes shared among all the alarms until they exit this local scope. The issue is when they exit this scope the alarms split off once again into individual entities and the "inherited" alarms stay in this loop and get stuck.
What I've also noticed is that each alarm still gets it's local property and they are all incremented but they don't exit this loop as one entity so they continue through my custom loop which rosterCounter variables that are out of bounds. My script accounts for this but it seems like everything gets scrambled after that.
I realize that this solution is not viable/robust for this issue, but I just wanted to give insight to what I've been trying up until now.
Is there a solution to this that I'm not seeing? I feel that the request to not continue down the roster after a user acknowledges an alarm/group of alarms isn't outlandish but doesn't seem possible with the tools I'm given without some clever workaround.
Please let me know if anyone has further questions about my current setup. Any input is appreciated. Thank you for your time.