Expression Structure Script to Replace Spaces with Underscore

I have the following transform script but it's only replacing spaces with underscores at the top level of the object structure and not any of the sub-levels.

def transform(self, value, quality, timestamp):
    # Create a new dictionary to store the transformed keys
    transformed_value = {}

    # Define a helper function to recursively transform the keys
    def transform_keys(dictionary):
        # Iterate through each key-value pair in the dictionary
        for key, val in dictionary.items():
            # Replace spaces with underscores in the key
            new_key = key.replace(" ", "_")
            
            # Check if the value is a dictionary
            if isinstance(val, dict):
                # Recursively transform the sub-dictionary
                transformed_value[new_key] = transform_keys(val)
            else:
                # Add the new key and corresponding value to the transformed dictionary
                transformed_value[new_key] = val

    # Call the helper function to transform the keys
    transform_keys(value)
    
    # Return the transformed dictionary
    return transformed_value

Any thoughts on what the issue is or how to modify this to evaluate correctly through all the sub-levels and replace spaces with underscores?

You're close, just move transformed_value inside your inner function and return it at the end.

def transform_keys(dictionary):
    transformed_value = {}
    
    for key, val in dictionary.items():
        new_key = key.replace(" ", "_")
        
        if isinstance(val, dict):
            transformed_value[new_key] = transform_keys(val)
        else:
            transformed_value[new_key] = val

    return transformed_value

Then at the end of the script:

return transform_keys(value)
1 Like

Here is the updated script but it didn't change anything.

def transform(self, value, quality, timestamp):
        
    # Define a helper function to recursively transform the keys
    def transform_keys(dictionary):
        # Create a new dictionary to store the transformed keys
        transformed_value = {}
        
        # Iterate through each key-value pair in the dictionary
        for key, val in dictionary.items():
            # Replace spaces with underscores in the key
            new_key = key.replace(" ", "_")
            
            # Check if the value is a dictionary
            if isinstance(val, dict):
                # Recursively transform the sub-dictionary
                transformed_value[new_key] = transform_keys(val)
            else:
                # Add the new key and corresponding value to the transformed dictionary
                transformed_value[new_key] = val
        
        # Return the transformed dictionary
        return transformed_value
    
    # Call the helper function to transform the keys
    return transform_keys(value)

This is the current result:
image
The keys under Hours didn't change.

Hmm, maybe if you did something like

if val.isMappingType():

instead.

guessing that whatever these inner objects are fail the isinstance check against built-in dict.

It didn't work.

I'd be printing the key and the type of val here

Eg

system.perspective.print('{} {}'.format(key, type(val)) 
1 Like

Figured it out. Those objects must not register as python dictionaries when passed into the transform() function, so I used this instead.

if hasattr(val, 'items'):

Thanks,

I'll suggest making a function that transform keys into something that ignition likes and putting it in your library.
I use this:

import re

from unicodedata import normalize, combining

class MalformedKeyError(Exception):
	pass

def remove_accents(s):
	# NFKD: https://fr.wikipedia.org/wiki/Normalisation_Unicode#NFKD
	nfkd_form = normalize('NFKD', s)
	return "".join(c for c in nfkd_form if not combining(c))

def make_key(s):
	"""
	Sanitize a string so it can be used as a key in an ignition object.
	
	If the first character is a digit, an underscore is added before it
	accents are removed
	leading and trailing whitespaces are removed
	other whitespaces are replaced with an underscore
	"""
	if isinstance(s, str):
		s = s.decode('utf-8')
	key = s.strip().lower()
	if not key:
		raise MalformedKeyError("Can't sanitize {!r} into a valid key".format(s)) 
	if key[0] in "0123456789":
		key = '_' + key
	return re.sub('\W+', '_', remove_accents(key))

But maybe it's overkill for your use case ?

I'd also put the "transform_keys" function in a script library, this way it won't have to be redefined every time the transform runs, and it's a useful function anyway that you might want to reuse at some point.

Here's what the whole thing would look like, with a slight rewrite of the inner function and a simple replace to keep things simple:

in a custom_lib script package:

def make_key(s):
	return s.replace(' ', '_')

def sanitize_dict_keys(dic):
	return {
		make_key(k): sanitize_dict_keys(v) if hasattr(v, 'items') else v
		for k, v in dic.iteritems()
	}

in your transform:

def transform(self, value, quality, timestamp):
	return custom_lib.sanitize_dict_keys(value)

edit:
This actually seemed like a useful enough function, so I added it to my own library:

def sanitize_dict_keys(dic, sfunc=make_key, recursive=True):
	"""
	Sanitize a dictionary keys.
	
	params:
		dic:		Dictionnary that will be sanitized
		sfunc:		A function that takes a string and returns a string, will be passed the keys of the dictionnary
		recursive:	A boolean that indicates wheter to transform the keys of nested dictionnaries or only the root one. default True
	
	return:
		The sanitized dictionnary.
	"""
	return {
		sfunc(k): sanitize_dict_keys(v, sfunc, recursive) if hasattr(v, 'items') and recursive else v
		for k, v in dic.iteritems()
	}
2 Likes

Thanks, I appreciate it. That will help a lot.