# Json Diffing Discussion

**URL:** <https://forum.inductiveautomation.com/t/json-diffing-discussion/96800>\
**Category:** Ignition\
**Created:** [October 2, 2024, 2:11pm UTC](https://forum.inductiveautomation.com/t/json-diffing-discussion/96800 "2024-10-02T14:11:40Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![pturmel](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pturmel/32/3295_2.png) [@pturmel](https://forum.inductiveautomation.com/u/pturmel)\
**Post date:** [October 2, 2024, 2:11pm UTC](https://forum.inductiveautomation.com/t/json-diffing-discussion/96800/1 "2024-10-02T14:11:40Z")

</div>

> [@New Feature - Tag Report Tool](https://forum.inductiveautomation.com/t/new-feature-tag-report-tool/61594/25):
>
> Make all json diff-friendly with tidyjson/jsonlint/whatever.
> 
> Use "version sort" or `AlphaNumComparator` so numbers in tag names are ordered the way humans like.

FWIW, I made a script recently that achieves this with JSON tag exports, and could be tweaked for many other cases (currently uses the "name" key for sorting lists of dictionaries).

```python
# Functions designed to be called via objectScript() in UI expressions.

from java.io import File
from java.lang import System, Throwable
from java.util import TreeMap
from com.inductiveautomation.ignition.common.util import Comparators

logger = system.util.getLogger(system.util.getProjectName() + '.' + system.reflect.getModulePath())
tkCtx = system.util.toolkitCtx()
ciAlnumCmp = Comparators.alphaNumeric(True)

def ordering(subject):
	if hasattr(subject, 'items'):
		subst = TreeMap(ciAlnumCmp)
		for k, v in subject.items():
			subst[k] = ordering(v)
		return subst
	first = None
	try:
		first = subject[0]
	except:
		pass
	try:
		if first and hasattr(first, 'items'):
			subst = TreeMap(ciAlnumCmp)
			subst.update([(x.get('name', str(i)), ordering(x)) for (i, x) in enumerate(subject)])
			return [x for x in subst.values()]
	except Throwable, t:
		logger.info("Java failure in ordering:", t)
	except Exception, e:
		logger.info("Jython failure in ordering:", system.reflect.asThrowable(e, None))
	return subject
#

#
def orderedJson(source):
	json = system.util.jsonDecode(source)
	ordered = ordering(json)
#	print repr(ordered)
	return system.util.jsonEncode(dict(_=ordered), 2)[6:-1]
#

```

Note that `ordering()` is a _ **recursive** _ deep copy operation, where dictionary-like types are replaced with a key-ordered dictionary, and lists of dictionaries are conditionally re-ordered by their inner `name` keys.

The use of `name` for lists of dictionaries could be made an argument to make this more widely applicable.

Consider this a plea for this algorithm or similar to become the default behavior whenever Ignition writes JSON to disk or for download/export.

The client who prompted this script's creation found it effective for re-synchronizing UDT definitions from dev to prod.

---

<div class="post-metadata">

**Author:** ![pturmel](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pturmel/32/3295_2.png) [@pturmel](https://forum.inductiveautomation.com/u/pturmel)\
**Post date:** [October 2, 2024, 2:47pm UTC](https://forum.inductiveautomation.com/t/json-diffing-discussion/96800/2 "2024-10-02T14:47:16Z")

</div>

I should mention that side-by-side or inline diff views of related tag JSON that has been ordered this way is simply beautiful. If looking for an abbreviated form to show scattered small changes, I recommend `diff -U20 file1 file2` so that the context around differences tends to include plenty of unique identifying information.

---

<div class="post-metadata">

**Author:** ![msteele](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/msteele/32/77889_2.png) [@msteele](https://forum.inductiveautomation.com/u/msteele)\
**Post date:** [October 2, 2024, 2:55pm UTC](https://forum.inductiveautomation.com/t/json-diffing-discussion/96800/3 "2024-10-02T14:55:51Z")

</div>

This script is a simple yet critical quality of life upgrade for our DevOps workflows. Here's a screenshot of diffing two ordered JSON outputs in Notepad++, something impossible to do before:

 ![image](https://us1.discourse-cdn.com/inductiveautomation/original/3X/8/3/83a41919a87d49fad880a41944f4961e68b9ea5d.png)

I urge IA to adopt this ordering for all JSON exports.

- Imagine an upgrade to Designer where Export JSON creates these ordered, diffable exports by default rather than having to route the current exports through a reordering tool.
- Imagine in 8.3 once things like our tag providers are in the filesystem natively, we can see meaningful changes inside our source code management tools.

---

<div class="post-metadata">

**Author:** ![nminchin](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/nminchin/32/6911_2.png) [@nminchin](https://forum.inductiveautomation.com/u/nminchin)\
**Post date:** [October 2, 2024, 5:12pm UTC](https://forum.inductiveautomation.com/t/json-diffing-discussion/96800/4 "2024-10-02T17:12:57Z")

</div>

I haven't tried this yet, but will this enable diffing of tag structures that have had tags added, eg new tags added to the top of a folder for example? If you try to diff this with the default json export, it'll end up trying to diff the new tags against old tags (assuming the tags end up sorting in the order of representation insode the folder) since diff tools don't understand the names are the primary keys.

---

<div class="post-metadata">

**Author:** ![pturmel](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pturmel/32/3295_2.png) [@pturmel](https://forum.inductiveautomation.com/u/pturmel)\
**Post date:** [October 2, 2024, 5:14pm UTC](https://forum.inductiveautomation.com/t/json-diffing-discussion/96800/5 "2024-10-02T17:14:58Z")

</div>

Looked like it worked correctly for my client. When using the command line, the `-U20` is probably required.

Diff tools are pretty aggressive at finding large blocks of unchanged text to "synchronize" against.

You should try it. (:

You can test in the script console with `readFileAsString()` and `writeFile()`.

---

<div class="post-metadata">

**Author:** ![nminchin](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/nminchin/32/6911_2.png) [@nminchin](https://forum.inductiveautomation.com/u/nminchin)\
**Post date:** [October 2, 2024, 5:22pm UTC](https://forum.inductiveautomation.com/t/json-diffing-discussion/96800/6 "2024-10-02T17:22:25Z")

</div>

I'll give it a go for sure when I get in the office. it's a tad late/early at the moment 😅

 ![Screenshot_20241003_025148_Chrome](https://us1.discourse-cdn.com/inductiveautomation/original/3X/9/c/9c81fa9c5d3b84b147f7878455af5f05345b1cb0.jpeg)

---

<div class="post-metadata">

**Author:** ![pturmel](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pturmel/32/3295_2.png) [@pturmel](https://forum.inductiveautomation.com/u/pturmel)\
**Post date:** [October 2, 2024, 5:23pm UTC](https://forum.inductiveautomation.com/t/json-diffing-discussion/96800/7 "2024-10-02T17:23:38Z")

</div>

You're worse than @Kevin.Herron. 🙄

---

<div class="post-metadata">

**Author:** ![Kevin.Herron](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/kevin.herron/32/7259_2.png) [@Kevin.Herron](https://forum.inductiveautomation.com/u/Kevin.Herron)\
**Post date:** [October 2, 2024, 5:24pm UTC](https://forum.inductiveautomation.com/t/json-diffing-discussion/96800/8 "2024-10-02T17:24:06Z")

</div>

When I'm online at that hour it's because I'm up early, never because I'm up late.

---

<div class="post-metadata">

**Author:** ![nminchin](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/nminchin/32/6911_2.png) [@nminchin](https://forum.inductiveautomation.com/u/nminchin)\
**Post date:** [October 2, 2024, 5:26pm UTC](https://forum.inductiveautomation.com/t/json-diffing-discussion/96800/9 "2024-10-02T17:26:04Z")

</div>

Same for me these days. This is me waking up at an abominable time rather than staying up

---

<div class="post-metadata">

**Author:** ![pturmel](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pturmel/32/3295_2.png) [@pturmel](https://forum.inductiveautomation.com/u/pturmel)\
**Post date:** [October 2, 2024, 5:29pm UTC](https://forum.inductiveautomation.com/t/json-diffing-discussion/96800/10 "2024-10-02T17:29:03Z")

</div>

> [@nminchin](#):
>
> an _ **abominable** _ time

Perfect adjective. If I'm on the forum before ~7am (breakfast with the forum), it's because I'm visiting Europe.

---

<div class="post-metadata">

**Author:** ![pturmel](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pturmel/32/3295_2.png) [@pturmel](https://forum.inductiveautomation.com/u/pturmel)\
**Post date:** [October 2, 2024, 9:55pm UTC](https://forum.inductiveautomation.com/t/json-diffing-discussion/96800/11 "2024-10-02T21:55:46Z")

</div>

> [@pturmel](#):
>
> that `ordering()` is a _ **recursive** _ deep copy operation

This part works as expected, yielding ordered dictionaries and lists. _ **But....** _

So I've been poking around with `org.json.JSONObject` and discovered that IA isn't shipping the version that is publicly posted on Github. Which is good, because the public code that generates the pretty-printed indented format is ugly and non-extensible and won't actually output dictionaries with ordered keys, and can't easily be made to do so. (The author has strong opinions on this.)

The version included with Ignition has override-able methods offering sorted keys out of the box (but not human-friendly). Which are, in fact, what is coming out of the above `jsonEncode()`. ☹

Any of the usual suspects care to point at the modifications?

I'd like to override to actually get the encoding in my desired key order, but it will be pointless if this is IA custom and v8.3 changes it.

---

<div class="post-metadata">

**Author:** ![Kevin.Herron](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/kevin.herron/32/7259_2.png) [@Kevin.Herron](https://forum.inductiveautomation.com/u/Kevin.Herron)\
**Post date:** [October 2, 2024, 10:00pm UTC](https://forum.inductiveautomation.com/t/json-diffing-discussion/96800/12 "2024-10-02T22:00:57Z")

</div>

The `org.json` stuff is pretty much only used in old/legacy code. We use gson in anything written in the last decade?

---

<div class="post-metadata">

**Author:** ![pturmel](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pturmel/32/3295_2.png) [@pturmel](https://forum.inductiveautomation.com/u/pturmel)\
**Post date:** [October 2, 2024, 10:02pm UTC](https://forum.inductiveautomation.com/t/json-diffing-discussion/96800/13 "2024-10-02T22:02:33Z")

</div>

Your SystemUtils bytecode suggests you are still using it for scripted encode/decode.

But I'm inclined to abandon it too, if that's what you've done.

---

<div class="post-metadata">

**Author:** ![Kevin.Herron](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/kevin.herron/32/7259_2.png) [@Kevin.Herron](https://forum.inductiveautomation.com/u/Kevin.Herron)\
**Post date:** [October 2, 2024, 10:03pm UTC](https://forum.inductiveautomation.com/t/json-diffing-discussion/96800/14 "2024-10-02T22:03:40Z")

</div>

Of course, those scripting functions have been around forever, and we're not going to swap out the underlying implementation because it will 100% cause a regression for somebody.

I think the plan is to add new `system.json` functions in 8.3 or 8.3.x?

---

<div class="post-metadata">

**Author:** ![pturmel](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pturmel/32/3295_2.png) [@pturmel](https://forum.inductiveautomation.com/u/pturmel)\
**Post date:** [October 2, 2024, 10:08pm UTC](https://forum.inductiveautomation.com/t/json-diffing-discussion/96800/15 "2024-10-02T22:08:16Z")

</div>

Ok. I'll patch up something that'll hold until the new ("reference") implementations appear. (Encode/Decode in expression functions has become **very** useful.)

Do any methods/constructors in the GSON implementations cooperate with ordered maps?

(I suppose I should just research that... But not today. My brain is scorched.)

---

<div class="post-metadata">

**Author:** ![Kevin.Herron](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/kevin.herron/32/7259_2.png) [@Kevin.Herron](https://forum.inductiveautomation.com/u/Kevin.Herron)\
**Post date:** [October 2, 2024, 10:09pm UTC](https://forum.inductiveautomation.com/t/json-diffing-discussion/96800/16 "2024-10-02T22:09:39Z")

</div>

GSON's `JsonObject` uses a `LinkedTreeMap` internally.

For maximal confusion you'll find that there is both google-namespaced GSON and an internal IA GSON on the class path.

---

<div class="post-metadata">

**Author:** ![pturmel](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pturmel/32/3295_2.png) [@pturmel](https://forum.inductiveautomation.com/u/pturmel)\
**Post date:** [October 3, 2024, 2:30pm UTC](https://forum.inductiveautomation.com/t/json-diffing-discussion/96800/17 "2024-10-03T14:30:29Z")

</div>

Any chance the JavaDoc for `com.inductiveautomation.ignition.common.gson` could be included in the public SDK docs?

I'll probably want the artifact IDs before too long...

---

<div class="post-metadata">

**Author:** ![Kevin.Herron](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/kevin.herron/32/7259_2.png) [@Kevin.Herron](https://forum.inductiveautomation.com/u/Kevin.Herron)\
**Post date:** [October 3, 2024, 2:49pm UTC](https://forum.inductiveautomation.com/t/json-diffing-discussion/96800/18 "2024-10-03T14:49:41Z")

</div>

Mmm, probably not in the SDK docs, this is a separately published artifact that doesn't appear to be published with javadoc JARs. We did publish source JARs, but I think those get stripped away when accessing the public nexus proxy.

We should probably publish it to the thirdparty repo instead. @Brian_Ray might know more.

---

<div class="post-metadata">

**Author:** ![pturmel](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/pturmel/32/3295_2.png) [@pturmel](https://forum.inductiveautomation.com/u/pturmel)\
**Post date:** [October 3, 2024, 2:54pm UTC](https://forum.inductiveautomation.com/t/json-diffing-discussion/96800/19 "2024-10-03T14:54:34Z")

</div>

> [@Kevin.Herron](#):
>
> We did publish source JAR

Ah! I see it here:

> **[GitHub - inductiveautomation/gson: Fork of Google's GSON that makes JsonElement...](https://github.com/inductiveautomation/gson)**
>
> Fork of Google's GSON that makes JsonElement implement Serializable

Thanks, this helps.

---

<div class="post-metadata">

**Author:** ![Kevin.Herron](https://sea1.discourse-cdn.com/inductiveautomation/user_avatar/forum.inductiveautomation.com/kevin.herron/32/7259_2.png) [@Kevin.Herron](https://forum.inductiveautomation.com/u/Kevin.Herron)\
**Post date:** [October 3, 2024, 3:02pm UTC](https://forum.inductiveautomation.com/t/json-diffing-discussion/96800/20 "2024-10-03T15:02:12Z")

</div>

Hopefully that lines up... that may be an older version. We have another one on our internal GitHub that is newer.

[Next page](https://forum.inductiveautomation.com/t/json-diffing-discussion/96800.md?page=2)