Best way to get JSON config in perspective

Ignition
1

I have a need to store between 10-50 different JSON objects (approx. 5000 characters in plain text)

Currently they are stored in a database, and joined against the data they are relevant too. This is fine except when I query the data I am ending up repeating the JSON for every record and the dataset becomes quite large.

I know that I can store it as dictionaries in a project script, a web dev text resource, a general file on the gateway, and I’m sure there are more options.

However I’m curious which version would be the most optimal… the text resource sounds good, but to access it I would have to make an HTTP call as opposed to just opening it as a file.

Any ideas are welcome!

2

Text resource would be the slowest as you’ll then be reading it from disk rather than from memory.
Stroing as array(s) of dictionaries would be the fastest I think. It’s fun making this globally available though, as I discovered. There’s an independent scope for designer, tags, gateway scripts… Maybe more

3

So would you store them as a text resource, and then have a gateway script that stores them as globals every 60 seconds or so?

4

I would cache them in a dictionary nested in the dictionary from system.util.getGlobals(). Update in place on demand. In a script named jsonCache, you can reliably set that up with something like this:

c = system.util.getGlobals().setdefault('jsonCache', {})

Then, wherever you need it, just use jsonCache.c.get('someKey'). If it returns None, retrieve/decode it from permanent storage, and assign back to the cache for future users: jsonCache.c['someKey'] = json. Jython dictionaries are thread-safe, so parallel updaters won’t break anything.

If the retrieval is sufficiently generic for any given key, you could make that generic with something like this (also in jsonCache):

def get(key):
    json = c.get(key)
    if json is not None:
        return json
    # Retrieve from permanent storage
    # into "json".
    c[key] = json
    return json

Then users only need jsonCache.get('someKey').

3 Likes
5

I keep reading that as "get smokey" :sweat_smile:

2 Likes
6

It’s managed with the rest of the project through git, any ideas on how you’d manage a diff between the one existing in the cache vs and potential new ones? That’s kind’ve where I was thinking about a timer script to verify if it’s different.

7

I wouldn’t bother with diff. Just cache the file’s stat() items (mod time, filesize, inode, etc) and reload on change. If the content is still equal, it won’t hurt anything to just replace the dictionary entry unconditionally. If some event happens where you know there is a change, you can pre-emptively del that entry to force a reload earlier than a timed check. (Do not check stat() on every lookup–that’ll kill your cache performance.)

1 Like