Just wondering if anyone’s written a JSON compare tool for comparing Views and Tags?
The generic JSON compare tools online are a bit useless for Ignition.
I want be able to see what’s changed between different versions of the project, and it would be particularly useful to show all the differences between an offline dev and live system as a way to catch any missed resources when deploying new changes, especially larger projects developed offline.
Is the way the json structure\order for views consistent? Almost seems like you’d just want to use github to track changes.
Yes it’s consistent. Github will be able to tell you all of the changes made, but not in a friendly way. It won’t tell you “these tags were added, these tags were deleted, these tags had these properties change value, etc.”
It’ll tell you “this part of the json is different, this part was removed” but it won’t be able to associate it to a tag which is the uid for us
I wrote something over the weekend in Python to compare tags. It works pretty good! Although I’m finding that there are a lot of properties deleted between tags that were upgraded by a gateway upgrade and the same tags exported from v7 as XML and imported into the v8 system. Lots of properties like “tagGroup”, “readOnly”, “read/writePermissions” and their sub-objects.
I also found that migration from v7 to v8 doesn’t migrate UDT parameters across with their correct dataType. All of my UDT definitions use String-type parameters, however all of these parameters were converted to Integer. The v7 XML export/import to v8 had the correct String dataType.
I ran the tool over a UDT that had some minor changes made to it by another engineer (we change-freezed to upgrade, but urgent on-site work meant that changes had to be made, so they noted which resources they’re made changes to so I could migrate them back into the upgraded project), and it picked up 1400 or so changes. However, when I ran through them, I found there were only 2 modifications that were actually useful, with the rest being all of the other things mentioned above. I assume it will report more useful things however once only working in v8.
I recently rewrote my tag compare script which was a bit dopey, to use a Python library littletree I found via StackOverflow. Most of the kudos goes to its developer who answered my SO question with the majority of the code, but I modified it to include more detail and spit out a CSV.
This will only run in CPython, I use PyCharm as my IDE. It requires a min of Python 3.10 (dependency of the littletree library).
The only library dependency not included in the standard libraries is littletree (pip install littletree)
from littletree import Node # littletree requires minimum Python 3.10
import json
def compare_tag_json(json_orig_filepath, json_new_filepath, exclude_tags_in_removed_folders=True, exclude_tags_in_added_udt_instances=True):
"""
This is a wrapper for function compare_tag_obj.
Takes two tag JSON exports to files and produces a report of changes between the tag structures, including:
- additions/deletions of tags
- additions/modifications/removal of tag props
Args:
json_orig_filepath:
The filepath to the original tag json file for the comparison.
json_new_filepath:
The filepath to the new tag json file for the comparison.
exclude_tags_in_removed_folders:
For folders removed in the new file, this option will exclude the tag changes (i.e. deletions) to the tags
contained within the folder.
Default: True
exclude_tags_in_added_udt_instances:
For UDT Instances added in the new file, this option will exclude the tag changes (i.e. additions) to the
tags contained within the UDT Instance.
Default: True
Returns:
The table of changes in CSV format with headers: tagpath, tagtype, change, from, to
Also prints the results as a CSV to the console.
"""
with open(json_orig_filepath, 'r') as f:
json_orig = f.read()
with open(json_new_filepath, 'r') as f:
json_new = f.read()
original = json.loads(json_orig)
modified = json.loads(json_new)
return compare_tag_obj(original, modified, exclude_tags_in_removed_folders, exclude_tags_in_added_udt_instances)
def compare_tag_obj(tags_original, tags_modified, exclude_tags_in_removed_folders=True, exclude_tags_in_added_udt_instances=True):
"""
Takes two tag JSON as Py lists/dicts and produces a report of changes between the tag structures, including:
- additions/deletions of tags
- additions/modifications/removal of tag props
Args:
tags_original:
The list/dict structure of the original tag json file for the comparison.
tags_modified:
The list/dict structure of the new tag json file for the comparison.
exclude_tags_in_removed_folders:
For folders removed in the new file, this option will exclude the tag changes (i.e. deletions) to the tags
contained within the folder.
Default: True
exclude_tags_in_added_udt_instances:
For UDT Instances added in the new file, this option will exclude the tag changes (i.e. additions) to the
tags contained within the UDT Instance.
Default: True
Returns:
The table of changes in CSV format with headers: tagpath, tagtype, change, from, to
Also prints the results as a CSV to the console.
"""
original_tree = Node.from_dict(tags_original, identifier_name='name', children_name='tags')
modified_tree = Node.from_dict(tags_modified, identifier_name='name', children_name='tags')
# Collect changes in a list
changes = []
for diff_node in original_tree.compare(modified_tree).iter_tree():
diff_data = diff_node.data
tagpath = str(diff_node.path)[1:]
if not diff_data:
continue # Data was the same
if 'self' not in diff_data:
if not exclude_tags_in_added_udt_instances or not any(change['tagpath'] in tagpath and change['change'] == 'Tag added' and change['tagtype'] == 'UdtInstance' for change in changes):
changes.append({'tagpath': tagpath,
'tagtype': diff_data['other']['tagType'],
'change': 'Tag added',
'from': None,
'to': None})
elif 'other' not in diff_data:
if not exclude_tags_in_removed_folders or not any(change['tagpath'] in tagpath and change['change'] == 'Tag removed' for change in changes):
changes.append({'tagpath': tagpath,
'tagtype': diff_data['self']['tagType'],
'change': 'Tag removed',
'from': None,
'to': None})
else:
original_data, modified_data = diff_data['self'], diff_data['other']
all_keys = set(list(original_data.keys()) + list(modified_data.keys()))
for key in all_keys:
if key in original_data and key in modified_data:
if original_data[key] != modified_data[key]:
changes.append({'tagpath': f'{tagpath}.{key}',
'tagtype': original_data['tagType'],
'change': 'Modified property',
'from': original_data[key],
'to': modified_data[key]
})
elif key in original_data:
changes.append({'tagpath': f'{tagpath}.{key}',
'tagtype': original_data['tagType'],
'change': 'Removed property',
'from': original_data[key],
'to': None
})
elif key in modified_data:
changes.append({'tagpath': f'{tagpath}.{key}',
'tagtype': modified_data['tagType'],
'change': 'Added property',
'from': None,
'to': modified_data[key]})
changes_csv = '"' + '","'.join(changes[0].keys()) + '"\r\n'
changes_csv += '\r\n'.join('"' + '","'.join(map(str, change.values())) + '"' for change in changes)
print(changes_csv)
return changes_csv
The result is a table like this:
| tagpath | tagtype | change | from | to |
|---|---|---|---|---|
| A/New Tag 1.value | AtomicTag | Added property | None | 1 |
| A/New Instance | UdtInstance | Tag removed | None | None |
| A/New Tag 2 | AtomicTag | Tag removed | None | None |
| A/New Tag.value | AtomicTag | Modified property | 2 | 1 |
| A/Folder B | Folder | Tag added | None | None |
| A/New Instance 3 | UdtInstance | Tag added | None | None |
or this:
| tagpath | tagtype | change | from | to |
|---|---|---|---|---|
| Analogue Input 01.parameters | UdtType | Modified property | {'Alarm_ParentDevice': {'dataType': 'String'}, 'Unit': {'dataType': 'String', 'value': ''}, 'Alarm_Area': {'dataType': 'String'}, 'Format': {'dataType': 'String', 'value': '#,##0.0'}, 'PLCName': {'dataType': 'String', 'value': {'bindType': 'parameter', 'binding': '{PLCName}'}}, 'Description': {'dataType': 'String', 'value': ''}, 'DeviceName': {'dataType': 'String', 'value': ''}, 'Global.': {'dataType': 'String', 'value': ''}} | {'Alarm_ParentDevice': {'dataType': 'String'}, 'Unit': {'dataType': 'String'}, 'Alarm_Area': {'dataType': 'String'}, 'Format': {'dataType': 'String', 'value': '#,##0.0'}, 'PLCName': {'dataType': 'String', 'value': {'bindType': 'parameter', 'binding': '{PLCName}'}}, 'Description': {'dataType': 'String', 'value': ''}, 'DeviceName': {'dataType': 'String', 'value': ''}, 'Global.': {'dataType': 'String', 'value': ''}} |
| Analogue Input 01/Low Alarm SP | AtomicTag | Tag removed | None | None |
| Analogue Input 01/Alarms/High | AtomicTag | Tag removed | None | None |
| Analogue Input 01/Alarms/_AnyActive.executionMode | AtomicTag | Removed property | EventDriven | None |
| Analogue Input 01/Alarms/_AnyActive.expression | AtomicTag | Removed property | {[.]High High} | |
| Analogue Input 01/Alarms/_AnyActive.opcItemPath | AtomicTag | Modified property | {'bindType': 'parameter', 'binding': 'ns=1;s=[{PLCName}]{Global.}{DeviceName}.Alarms'} | {'bindType': 'parameter', 'binding': 'ns=1;s=[{PLCName}]{Global.}{DeviceName}.AnyAlarm'} |
| Analogue Input 01/Alarms/Low Low | AtomicTag | Tag removed | None | None |
| Analogue Input 01/Alarms/Low | AtomicTag | Tag removed | None | None |
| Analogue Input 01/Alarms/Comms Fault | AtomicTag | Tag removed | None | None |
| Analogue Input 01/Alarms/Signal Fault | AtomicTag | Tag added | None | None |
| Analogue Input 01/Alarms/Over or Under Range | AtomicTag | Tag added | None | None |
| Analogue Input 01/Simulation | AtomicTag | Tag removed | None | None |
| Analogue Input 01/High Alarm SP | AtomicTag | Tag removed | None | None |
| Analogue Input 01/Low Low Alarm SP | AtomicTag | Tag removed | None | None |
| Analogue Input 01/Low Alarm Time | AtomicTag | Tag removed | None | None |
| Analogue Input 01/High High Alarm Time | AtomicTag | Tag removed | None | None |
| Analogue Input 01/High Alarm Time | AtomicTag | Tag removed | None | None |
| Analogue Input 01/Low Low Alarm Time | AtomicTag | Tag removed | None | None |
| Analogue Input 01/High High Alarm SP | AtomicTag | Tag removed | None | None |
original = {
'name': 'A',
'tagType': 'Folder',
'tags': [
{
'valueSource': 'memory',
'name': 'New Tag 1',
'tagType': 'AtomicTag'
},
{
'name': 'New Instance',
'typeId': 'Alarm Summary',
'parameters': {
'MainTagPath': {
'dataType': 'String',
'value': 'Bob'
},
'Description': {
'dataType': 'String',
'value': 'Descrp'
}
},
'tagType': 'UdtInstance',
'tags': [
{
'name': 'Area PLCs for Alarm Reset',
'tagType': 'AtomicTag'
},
{
'name': 'Active Ack',
'tagType': 'AtomicTag'
},
{
'name': 'Active Unack',
'tagType': 'AtomicTag'
},
{
'name': 'Clear Unack',
'tagType': 'AtomicTag'
}
]
},
{
'valueSource': 'memory',
'name': 'New Tag 2',
'tagType': 'AtomicTag'
},
{
'name': 'New Instance 1',
'typeId': 'Alarm Summary',
'parameters': {
'MainTagPath': {
'dataType': 'String',
'value': 'Bob'
},
'Description': {
'dataType': 'String',
'value': 'Descrp'
}
},
'tagType': 'UdtInstance',
'tags': [
{
'name': 'Active Ack',
'tagType': 'AtomicTag'
},
{
'name': 'Area PLCs for Alarm Reset',
'tagType': 'AtomicTag'
},
{
'name': 'Clear Unack',
'tagType': 'AtomicTag'
},
{
'name': 'Active Unack',
'tagType': 'AtomicTag'
}
]
},
{
'valueSource': 'memory',
'name': 'New Tag',
'value': 2,
'tagType': 'AtomicTag'
}
]
}
modified = {
'name': 'A',
'tagType': 'Folder',
'tags': [
{'name': 'Folder B',
'tagType': 'Folder'
},
{
'valueSource': 'memory',
'name': 'New Tag',
'value': 1,
'tagType': 'AtomicTag'
},
{
'valueSource': 'memory',
'name': 'New Tag 1',
'value': 1,
'tagType': 'AtomicTag'
},
{
'name': 'New Instance 3',
'typeId': 'Alarm Summary',
'parameters': {
'MainTagPath': {
'dataType': 'String',
'value': 'Bob'
},
'Description': {
'dataType': 'String',
'value': 'Descrp'
}
},
'tagType': 'UdtInstance',
'tags': [
{
'name': 'Area PLCs for Alarm Reset',
'tagType': 'AtomicTag'
},
{
'name': 'Clear Unack',
'tagType': 'AtomicTag'
},
{
'name': 'Active Unack',
'tagType': 'AtomicTag'
},
{
'name': 'Active Ack',
'tagType': 'AtomicTag'
}
]
}
]
}
compare_tag_obj(original, modified, exclude_tags_in_removed_folders=True, exclude_tags_in_added_udt_instances=True)
Hi,
How can I run your code?
do you have any example?
thank you
Nick was/is using PyCharm to execute this code, but I would expect any Python 3.1+ environment would be able to run it, so pick your poison.
Note: It will not run inside of an Ignition Designer or Client without some fancy footwork to execute it externally.
yes, I am running it within VS code, I installed the litletree library
and I am calling the function passing the path of my Json.
compare_tag_json("C:\Users\l.abdallah\Desktop\original.json", "C:\Users\l.abdallah\Desktop\modified.json", exclude_tags_in_removed_folders=True, exclude_tags_in_added_udt_instances=True)
and I don't have any results.
\ in python is an escape character, you need to escape it 'c:\\...'or use a raw string with r'c:\... '
However you should have got an error?
Definitely still is 
it's very charming. I find it far better than vs code for python. Whoever writes these smart IDEs must be gods. The amount of work that goes into them has to be insane
Edit:
as of my original writing, the littletree library requires min Python 3.10 to run
I'm trying to run this script now and I'm getting this error:
File "C:\Users\myUserName\Documents\Python Scripts\Ignition_JSON_TagDiff.py", line 73, in <module>
compare_tag_json('C:\\Users\\myUserName\\Documents\\Projects\\Project1\\Tag Backups\\Unsorted\\tags_20241028.json','C:\\Users\\myUserName\\Documents\\Projects\\Projec2\\Tag Backups\\Unsorted\\tags_20241028.json')
~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\myUserName\Documents\Python Scripts\Ignition_JSON_TagDiff.py", line 14, in compare_tag_json
compare_tag_obj(original, modified, exclude_tags_in_removed_folders, exclude_tags_in_added_udt_instances)
~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\myUserName\Documents\Python Scripts\Ignition_JSON_TagDiff.py", line 22, in compare_tag_obj
for diff_node in original_tree.compare(modified_tree).iter_tree():
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'iter_tree'
It looks like the objects have data in them. I have a space in my file path so I was worried that it wasn't opening the JSON files correctly, but when I use the debugger, I can see data in the original_tree and modified_tree objects. Any suggestions what might be causing this?
btw thanks for creating this code, I assumed doing a diff on two JSON tag exports would be simple, but everything is shuffled out of order.
I would need some tag exports to run this against to see what the errors are produced by, you can send via PM if you want. And also a copy of the script you're using, as mine wasn't compatible with running directly from the console.
I realise also that I'm missing docstrings... oops. I edited to add docstrings
That's very kind of you to offer that. I might take you up on that.
However, in the meantime, I ended up writing a Jython script that uses system.tag.browse with a recursive filter to get a list of tags (these are alphabetical!), and then I enumerate through the results, write the fullPath string to a JSON file, pass in the fullPath value from each result to system.tag.exportTags (with recursive set to false), and then write that string to the JSON file as well. There's a decent bit of formatting in the script to create a valid JSON file and generalize some of the data.
Essentially, I needed to compare two different projects' tags. In theory these two projects should be very similar but we needed to understand what differences exist. They are nearly identical, which is why I was surprised that just using "Tag Export" in the Designer was giving wildly different ordering. I'm not sure in what order they get exported. 
Here's a link to another post showing the method I use (that thread has a few more elegant options to suit their needs as well):
My procedure is as follows:
A sample output is also in that thread, except, the comparison was not from a system.tag.getConfiguration() output.
That deepSort() function could probably be enhanced to utilize an OrderedDict, but has suited my needs so far.
Just keep in mind that any diffing of tag json with your standard diff tools is going to have limited use-cases, ie reasonably simple cases. Even with that simple case you screenshotted here Json Diffing Discussion - #36 by Chris_Bingham, it's highlighting changes that shouldn't be there. A new tag was create called Donkey while another tag Detail was deleted, but diff tools tell you there are differences between those two tags. That's but am extremely simple example. I'm sure you can imagine the mess this could be with a much larger tag structure.
More details here Json Diffing Discussion - #32 by nminchin
I like to use the VSCode extension "Compare Folders" for this. I'll typically compare the root directory of my "staging" project with the root directory of the active "production" project.
