"Shelve" alarm for 5 minutes after notification

In an alarm pipeline, you can directly add a Delay block before your Notification block:
https://docs.inductiveautomation.com/pages/viewpage.action?pageId=72419404#PipelineBlocks-DelayBlock

Or, if you think there's a chance multiple alarms could enter the same pipeline within a given timespan, you could set up Consolidation on the notification block:
https://docs.inductiveautomation.com/display/DOC81/Notification+Block#NotificationBlock-Consolidation