Saving user configuration property object to SQL nvarchar and retrieving it again

I'm suffering from Engineer's Block. (Well, if writer's can suffer their version then why can't we. Try it on your boss.)

My application has some user configuration checkboxes linked to a `view.custom.mainTableShowColumns' and (the abbreviated version) looks like

{
  "process": true,
  "notes": true,
  "meta": false
}
  1. How do I script to convert that to a string to store in an MSSQL nvarchar column?
  2. How do I get it back to load the user's preferences back into `view.custom.mainTableShowColumns'?

My attempts so far either have <ObjectWrapper> and Unicode 'u' designators attached, or else store the qualified values of each of the object members.

Many thanks.

With MSSQL I used a NVARCHAR(MAX) column and system.util.jsonEncode to save to the db and system.util.jsonDecode when retrieving it out and it works well for me so far. That’s for me saving a python dictionary - I dont’ know if there are any special gotchas in your situation here, I know perspective objects are all wrapped up in stuff but if you’re able to get it into a dictionary and go from dictionary to the perspective object - I think the above should be fine.

If it is static ie always just three column process/notes/meta - I would opt for a column per field though. Always nice to ensure data integritywhen possible. This would also make probably make retrieval simpler.

2 Likes

What Brian said.
You may need to un-qualify the Perspective prop to a jython dict to feed to .jsonEncode().

1 Like

Thanks, Phil. Unfortunately your Integration Toolkit is not available on this customer's gateway. What's the long-hand approach to achieve the same result?

I understand, but in this case I'm trying to make it a generic user preference storage so there could be all sorts of stuff in there.

Thanks.

Recursive function, similar to this:

from java.util import Map
from com.inductiveautomation.ignition.common.model.values import QualifiedValue
try:
	pHook = system.util.toolkitCtx().getModule('com.inductiveautomation.perspective')
	objWrapper = pHook.class.classLoader.loadClass("com.inductiveautomation.perspective.gateway.script.PropertyTreeScriptWrapper$ObjectWrapper")
	mappings = (dict, Map, objWrapper)
except:
	mappings = (dict, Map)

# Strip qualified values out of hierarchical entities.
def deep(x):
	if x is None:
		return None
	if isinstance(x, QualifiedValue):
		return deep(x.value)
	if isinstance(x, basestring):
		return x
	if isinstance(x, mappings):
		x = dict(x)
		retv = dict()
		for k, v in x.items():
			retv[k] = deep(v)
		return retv
	try:
		return [deep(v) for v in x]
	except:
		return x

You'll need to use another method to get the gateway context.

(I wrote the above for use in the tag report utility in the Exchange, before I added the scripting and expression unQualify() functions.)

I'm doing something similar to that for storing some trend related properties per user. You're welcome to try and decipher and see if this helps you. This packages the config into a message handler, sends it to a remote gateway, and executes a namedquery to save/load it on the other gateway's database.

Fair warning - AI supported coding, so may not be best practices but it currently works well for me.

def savequickpoints(UserName, Data):
	"""
	Saves the quickpoints dataset to the database as a JSON string.
	"""
	# 1. Build the list from the dataset
	points_list = []
	
	if Data is not None:
		py_data = system.dataset.toPyDataSet(Data)
		
		for row in py_data:
			points_list.append({
				"Pad": row["Pad"],
				"Pen Name": row["Pen Name"],
				"Description": row["Description"],
				"Path": row["Path"]
			})

	# 2. Encode to JSON
	# THIS IS THE CRITICAL STEP. 
	# We must convert the list of objects into a text string.
	json_string = system.util.jsonEncode(points_list)

	# DEBUG: Print this to the console to confirm it looks like text, not "Dataset [...]"
	print "Saving the following string to SQL:", json_string

	# 3. Call the remote data handler
	return get_remote_data(
		query_path="custom_trend/savequickpoints",
		database="-redacted-",
		UserName = UserName,
		Data = json_string 
	)	

def loadquickpoints(UserName):
	"""
	Fetches the JSON string from SQL and converts it to a Dataset.
	"""
	# 1. Fetch the data
	result = get_remote_data(
		query_path="custom_trend/loadquickpoints",
		database="-redacted-",
		UserName = UserName
	)

	# 2. Sanitize the Result (Handle Dataset vs String)
	json_string = ""
	
	if result is None:
		return None
	
	# Check if result is an Ignition Dataset (common if Query Type is 'Select')
	if hasattr(result, 'getValueAt'):
		if result.getRowCount() > 0:
			json_string = result.getValueAt(0, 0) 
		else:
			return None
	else:
		# Assume it is already the scalar string
		json_string = result

	# 3. Decode JSON
	if not json_string:
		return None

	try:
		data_structure = system.util.jsonDecode(json_string)
	except:
		# Log the error if needed, or just return None
		return None

	# 4. Normalize to List
	# This prevents the 'unicode' error by ensuring we are looping over a list of objects,
	# not iterating over the keys of a single dictionary.
	if isinstance(data_structure, dict):
		data_list = [data_structure]
	elif isinstance(data_structure, list):
		data_list = data_structure
	else:
		return None # Unknown format

	# 5. Convert to Dataset
	headers = ["Pad", "Pen Name", "Description", "Path"]
	rows = []
	
	for item in data_list:
		# Critical Check: ensure the item is a dictionary before calling .get()
		if isinstance(item, dict):
			rows.append([
				item.get("Pad"),
				item.get("Pen Name"),
				item.get("Description"),
				item.get("Path")
			])
			
	return system.dataset.toDataSet(headers, rows)

The named queries are just:

loadquickpoints:
SELECT Data
FROM custom_quickpoints
WHERE UserName=:UserName
savequickpoints:
IF EXISTS (SELECT 1 FROM custom_quickpoints WHERE UserName = :UserName)
BEGIN
    -- User exists: Update their existing record
    UPDATE custom_quickpoints
    SET Data = :Data
    WHERE UserName = :UserName
END
ELSE
BEGIN
    -- New User: Insert a new record
    INSERT INTO custom_quickpoints (UserName, Data)
    VALUES (:UserName, :Data)
END

Edit: The on change script that is attached to session.custom.quickpoints:

def valueChanged(self, previousValue, currentValue, origin, missedEvents):
	UserName = self.props.auth.user.userName
	Data = self.custom.quickpoints
	if UserName:
	    remote_utils.savequickpoints(UserName, Data)

nvarchar(max) will work perfectly fine. Since you're using MSSQL however, you can consider using its JSON datatype which is optimized in different ways.