I am collecting backups for several Ignition Gateway servers via EAM. I am able to see the list of backups, but I do not see any way to purge old backups? Is there something somewhere what I am missing? It would be nice to only keep the last xx Days and purge everything else.
Thanks
Yeah, the EAM doesn’t currently have that functionality. It’s a great candidate for http://ideas.inductiveautomation.com though. On the EAM controller, you can manually set the archive path to a specified folder. This makes it easier to build an automated script that can remove gateway backup files older than XX days.
I built a user screen to display, prune and fetch EAM backups from the EAM gateway. There are four button scripts, and two client gateway event scripts and two client gateway scripts. If your interested send me a note and I can send you the project. I was going to post the code here but it just got to long.
To get a idea of who it works the screen looks like this;
My approach was to write a global script that deletes all the backups except for the number of backups that I choose to keep.
I execute this script as a tag event script on an expression tag that runs nightly an hour before the scheduled agent task for backup runs. You could also run it on schedule from a Gateway Event script.
def deleteOldBackupFiles():
################################################################################
# Purpose: To delete old backup files to prevent disk full condition on the server hard drive
#
# Change Management
# Date Who What
# ----------- --------------- ------------------------------------------------
# 22-Jan-2019 Lynn Martineau original version
################################################################################
# import libraries
import glob # used to get directory and file names
import os # used to delete files and remove directories
# initialize variables
backupFilePath = system.tag.read( 'Config/BackupFilePath' ).value
backupRetentionCount = system.tag.read( 'Config/BackupRetentionCount' ).value
backupFileMask = system.tag.read( 'Config/BackupFileMask' ).value # this prevents deleting non-backup files/folders if any
# initiate a logger
logger = system.util.getLogger( 'deleteOldFiles' )
# get a dataset of all registered agents
agentsDataset = shared.util.getAgentList() # custom function returns a dataset of Agents
# check to see if there are backup files on each Agent Gateway
for row in range( agentsDataset.rowCount ):
server = agentsDataset.getValueAt( row, 'Name' )
serverFilePath = backupFilePath + server
if system.file.fileExists( serverFilePath ):
# get a list of all of the backup folders for the Agent
backupFolders = glob.glob( serverFilePath + backupFileMask )
# create a dataset with all of the backup folders
dsHeader = ['backupPath']
dsData = []
for backupFolder in backupFolders:
dsData.append( [backupFolder] )
dsBackup = system.dataset.toDataSet( dsHeader, dsData )
# determine how many backup files to delete
backupsToDelete = max(dsBackup.rowCount - backupRetentionCount, 0)
# sort the backup folders oldest to newest
dsBackup = system.dataset.sort( dsBackup, 0, True )
# delete files in backup folders then remove the backup folders
for row in range( backupsToDelete ):
# get a list of the files in the backup folder
filesInBackupFolder = glob.glob( dsBackup.getValueAt( row, 0 ) + '\*' )
for fileInBackupFolder in filesInBackupFolder:
try:
# delete the file
os.remove( fileInBackupFolder )
# log the delete
logger.info( 'deleted file: %s' %fileInBackupFolder )
except:
# log a failure to delete the file
logger.info( 'failed to delete file: %s' %fileInBackupFolder )
# remove the backup folder
try:
# remove the backup folder
os.rmdir( dsBackup.getValueAt( row, 0 ) )
logger.info( 'removed backup folder: %s' %dsBackup.getValueAt( row, 0 ) )
except:
# log a failure to remove the backup folder
logger.info( 'failed to remove backup folder: %s' %dsBackup.getValueAt( row, 0 ) )
@martinel3746 I was trying to get your script for removing backups working and see you have a custom function that returns the agents on a gateway. Would you mind sharing the code that you are using for this?
Thanks!
We register unlicensed agents in EAM even though the connections fail when they are not in trial mode.
The EAM controller gives us a good place to manage them – we have over 130 last time I counted.
Here’s my function for getting only the licensed agents from the EAM Controller:
################################################################################
# Purpose: To get a list of the licensed Agent Gateways connected to this Controller
# Parameters: None
# Returns: dataset of licensed agents
# Change Management
# Date Who What
# ----------- ------------- ------------------------------------------------
# 03-Jul-2018 Lynn Martineau Original Version
# 18-Aug-2021 Lynn Martineau updated for Ignition 8.1
################################################################################
def getAgentList():
agentList = []
allAgents = system.tag.browseTags( '[System]Gateway/EAM/Agents' )
for agent in allAgents:
licensePath = '[System]%s/Independent/LicenseKey' %agent.path
licenseKey = system.tag.readBlocking([licensePath])[0].value
if licenseKey != None:
agentList.append( [agent.path, agent.name] )
agentDataset = system.dataset.toDataSet( ['Path','Name'], agentList )
agentDataset = system.dataset.sort( agentDataset, 'Name' )
return agentDataset
Just for kicks while I’m on topic, here’s a WebDev script that I use fairly regularly.
It gives me a quick list of servers without needing to visit the gateway:
# Access this service via:
# http://eamControllerName:8088/system/webdev/AgentManager/agents
import system.dataset
agentData = system.eam.queryAgentStatus()
htmlData = system.dataset.dataSetToHTML( True, agentData, 'EAM Agents' )
return {'html': htmlData}
@martinel3746 Thanks a million. Got this up and running this morning!
I like that script to return all agents from a web call. I was looking at building something similar with perspective.
I just updated the web client to add the agent’s IP address, an active link to the agent gateway, CPU Usage and Memory Usage.
# Access this service via:
# http://yourServerName:8088/system/webdev/yourProjectName/yourWebDevScriptName
import system.dataset
# get the basic EAM agent data into a dataset
agentData = system.eam.queryAgentStatus()
# initialize variables
tagPaths = []
cpuPaths = []
memPaths = []
# populate lists of tags to collect data about the agents
for row in range( agentData.rowCount ):
tagPaths.append( '[System]Gateway/EAM/Agents/%s/Independent/BrowserURL' %agentData.getValueAt(row,'AgentName') )
cpuPaths.append( '[System]Gateway/EAM/Agents/%s/Independent/Metrics/System/CPU Usage' %agentData.getValueAt(row,'AgentName') )
memPaths.append( '[System]Gateway/EAM/Agents/%s/Independent/Metrics/System/Memory Usage'%agentData.getValueAt(row,'AgentName') )
# read agent data tags
tagValues = system.tag.readBlocking( tagPaths )
cpuValues = system.tag.readBlocking( cpuPaths )
memValues = system.tag.readBlocking( memPaths )
# initialize lists for column inserts
ipList = []
linkList = []
cpuList = []
memList = []
# populate lists of agent data
for index in range( len(tagValues) ):
linkList.append( tagValues[index].value )
# parse to just the IP address - this is important for the text replacements to build the HTML link
ipList.append( tagValues[index].value.replace('http://','').replace(':8088','') )
cpuList.append( cpuValues[index].value )
memList.append( memValues[index].value )
agentData = system.dataset.addColumn( agentData, 1, ipList, 'IpAddress',str )
agentData = system.dataset.addColumn( agentData, 2, linkList, 'Link', str )
agentData = system.dataset.addColumn( agentData, 3, cpuList, 'CPU Usage',float )
agentData = system.dataset.addColumn( agentData, 4, memList, 'Mem Usage',float )
# convert the dataset to HTML
htmlData = system.dataset.dataSetToHTML( True, agentData, 'EAM Agents' )
# add HTML construct for the links.
# Note: if this is done prior to getting the htmlData, it all displays as text and will not function as a link
htmlData = htmlData.replace( 'http://', '<a href="http://' )
htmlData = htmlData.replace( '8088', '8088" target=_blank>Link</a>' )
return {'html': htmlData}
@tbackus I was just working on the same thing and thought I’d share my solution for anybody else that may still be having this issue. We are using EAM to gather gateway backups from all of our edge gateways. In order to purge the old backups, I created a batch file on the EAM Controller PC (it is running Windows 10) and scheduled it to run daily. The contents of the batch file are displayed below. It will delete anything within the C:\Ignition Backups folder and subfolders that has been last modified more than 14 days ago. I know this solution lies outside of the realm of Ignition, but it is easy to incorporate and it works well.
forfiles /p "C:\Ignition Backups" /s /m *.* /d -14 -c "cmd /c del @PATH"
Please vote on EAM Purge Backups.
Managing the backup files created by EAM should be managed via EAM configuration.