Copy dictionary and modify it without affecting the original

Good morning.

I have copied a dictionary and would like to modify the copy without affecting the original.
That’s easy when the dictionary elements are integers or strings, but if there are nested dictionaries, when I modify the copied dictionary, the original is modified.
For example (this is a simple example, actually I want to modify a “pen” (PowerChart) ):

As we can see in the image, the element [“a”] and [“b”], are only modified in “dic2”. That is expected. But the element [“c”][“c1”] is modified on “dic1” and “dic2”.
The only solution I’ve found is to copy [“c”] before modifying it, but it can be very tricky when you have to work with nested dictionaries, like the “pens” in a “PowerChart”.

Best Regards.

This answer appears to show my incomplete understanding of this topic. See further answers below.

Try deepcopy instead.

The manual points out that -

  • A shallow copy constructs a new compound object and then (to the extent possible) inserts references into it to the objects found in the original.
  • A deep copy constructs a new compound object and then, recursively, inserts copies into it of the objects found in the original.
2 Likes

image

image

… so I have created my own function to do deep copy of a dictionary.:

Now it works, thanks.

Sorry about that. Maybe deepcopy was added after Python 2.7 (which is what Igntion’s is based on). This question on StackOverflow asks about its use on 2.7.5 but there is no mention of a problem.

No, the OP tried to use deepcopy on a dict, which it doesn’t have. The correct form is to use the copy library, which has deepcopy()

import copy

dic1 = {'a':1, 'b':2, 'c':{'c1':3, 'c2':{'c21':20}}}
dic2 = copy.deepcopy(dic1)

print 'Antes de modificar'
print dic1
print dic2

print '------------'
print 'Después de modificar'

dic2['a'] = 2
dic2['b'] = 4
dic2['c']['c1'] = 6
dic2['c']['c2']['c21'] = 40

print dic1
print dic2

Output:

Antes de modificar
{'a': 1, 'b': 2, 'c': {'c1': 3, 'c2': {'c21': 20}}}
{'a': 1, 'b': 2, 'c': {'c1': 3, 'c2': {'c21': 20}}}
------------
Después de modificar
{'a': 1, 'b': 2, 'c': {'c1': 3, 'c2': {'c21': 20}}}
{'a': 2, 'b': 4, 'c': {'c1': 6, 'c2': {'c21': 40}}}
>>> 
4 Likes

Thanks, but it doesn’t work when I apply it on a “pen”:

Pen:
image

Script:

What is the type of penEjemplo ie system.perpective.print(str(type(penEjemplo)))? If its not a dict or a supported type for copy.deepcopy() I wouldn’t expect it to work.

2 Likes

As @bkarabinchak.psi pointed out even though the structure “looks” like a dict it isn’t actually a dict.
Actually the type of a perspective property when accessed from a script is:com.inductiveautomation.perspective.gateway.script.PropertyTreeScriptWrapper.ObjectWrapper.

Thus deepcopy chokes when it runs into an object that it doesn’t know how to handle.

If you need a true deepcopy, then you have couple of options.

  1. Write a function which recursively loops through the structure and correctly maps objects to dictionaries, and arrays to lists.
  2. Encode to JSON and de-encode to guarantee you’re dealing with native Python objects

I have done both of those things in the past.

Note: Option 1 will work for “perspective properties”, it doesn’t account for other possible types that aren’t handled by deepcopy. The assumption I have made here is that if it isn’t an object, or array, then it is a basic type (string,integer,float,etc…). The function can/should be modified to handle other types if you need that particular thing.

Code for option 1:

def recursiveCopy(original):
    from copy import deepcopy
    from com.inductiveautomation.ignition.common.script.abc import AbstractMutableJythonMap,AbstractMutableJythonSequence

    if isinstance(original,AbstractMutableJythonMap):
        return {key:recursiveCopy(value) for key,value in original.iteritems()}

    if isinstance(original,AbstractMutableJythonSequence):
        return [recursiveCopy(item) for item in original]

    return deepcopy(original)

Code for Option 2:

from copy import deepcopy
pen = deepcopy(system.util.jsonDecode(system.util.jsonEncode(self.view.custom.penEjemplo)))

There are reasons to go with option 1 over option 2, but for what it looks like you’re trying to do either will work.

I will say though, that if I were trying to modify a pen in a power chart, I would just modify the pen, not sure why you need a copy for that.

4 Likes

Yes, I have created my own function to deep copy.

I need to copy a “pen”, because from an example pen I dynamically create the “pens” of a PowerChart, only modifying the color, the name and the dataSource.

Thank you very much.

1 Like

Consider making your example pen a constant (dictionary) in a project script module. Then you can use deepcopy on it and modify as needed.

It is a good idea, but “example pen” is a param on a PowerChart.

I would suggest not having it as a param on your PowerChart.

There is nothing requiring you to have that parameter other than you. You can easily recreate the structure you need in a project script module and use it from there. Particularly if you plan to use this same method for more than one PowerChart component.

Use @lrose 's function instead.

But let’s try to work out how to get from your function to his.
First, let’s see what can be improved independently of context:

  • when iterating over a dict (let’s assume it’s called d), if you need the values, use
    for value in d.values()
    If you need the keys AND the values:
    for key, value in d.items()
  • when checking if a string contains a substring, don’t use find. Strings have methods just for this:
    – check if it’s anywhere in a string: "bar" in "foobarbaz"
    – check if a string starts with a substring: "foobarbaz".startswith("foo")
    – check if it ends with a substring: "foobarbaz".endswith("baz")
  • for type checking, use isinstance(obj, type)
    – you can check for several types at once: isinstance(obj, (dict, list))

which brings us to this version of the function:

def copiarProfundamente(dicOrigen, dicDestino):
    for k, v in dicOrigen.items():
        if isinstance(v, dict): 
            dicDestino[k] = v.copy()
            copiarProfundamente(v, dicDestino[k])
        else:
            dicDestino[k] = v

Much clearer, don’t you think ? But we’re just getting started.

Now about the function itself:

  • If your function should build a new object, let the function create and return it, don’t pass an empty object as parameter.
  • make the base case clear

This would look like something like this:

def deepcopy(obj):
	if isinstance(obj, dict):
		return {k: deepcopy(v) for k, v in obj.items()}
	return obj

So what happened there ?
The base case is the second return. If we don’t find a dict, we simply return the original object. Otherwise, call the function recursively for each key:value pair of the dict.
That’s a fundamental difference: We’re not iterating through the dict and using copy on it’s values that are also dict, then calling the function again. We’re calling the function directly on each keys, and their copy is handled by the base case. The return of that call is assigned to the keys of our new dict.

  • what if one of the value is a list ? lists are also not copied, but referenced. You can see it with a simple bit of code:
x = [[0] * 3] * 3
# x == [[0, 0, 0], [0, 0, 0], [0, 0, 0]]
x[0][0] = 1
# x == [[1, 0, 0], [1, 0, 0], [1, 0, 0]]
  • This means the function needs to be able to take in lists as well, not only dicts, if we want to be able to make recursive calls on lists

This is simple to fix: We apply the same thing we did to dicts, but to lists:

def deepcopy(obj):
	if isinstance(obj, dict):
		return {k: deepcopy(v) for k, v in obj.items()}
	if isinstance(obj, list):
		return [deepcopy(v) for v in obj]
	return obj

Now, we’re getting something similar to what @lrose suggested. The difference is that we’re doing this for dicts and lists, which copy.deepcopy already does. We’re basically re-implementing it.
The next jump is to replace the base case by a call to copy.deepcopy, and make the recursive calls conditions check for what copy.deepcopy can’t handle. Which brings us exactly to @lrose 's function:

def recursiveCopy(original):
    if isinstance(original,AbstractMutableJythonMap):
        return {key:recursiveCopy(value) for key,value in original.iteritems()}

    if isinstance(original,AbstractMutableJythonSequence):
        return [recursiveCopy(item) for item in original]

    return deepcopy(original)

(also note that obj.items() has been changed to obj.iteritems(), because this type of objects, which are not dicts, don’t have an .items() method)

One last thing, that I maybe should have started with: Your function works and deepcopy didn’t because you’re not actually feeding it a dict, but one of those type that deepcopy can’t handle.
By not checking the type of the initial object, you’re skipping straight to copying the dicts that it contains, and that are actuals dicts.
Which means you could have written your function like this (I think):

def copyWeirdIgnitionType(obj):
    return {k: deepcopy(v) for k, v in obj.iteritems()}
1 Like

I dont think that will work its doing the same as the default deepcopy minus one recursion.
You need the function with AbstractMutableJythonMap and AbstractMutableJythonSequence as @Irose provided. or use jsonDe-/Encode

Not quite, they actually do have a .items() method. However, I believe that best practice is to use .iteritems() in this case as it is a generator and doesn't build the full list as .items() does, reducing memory usage. Probably doesn't make a huge difference in most cases, but still better in the edge cases where there could be a large number of key value pairs.

Note: Outside of Ignition, where Python 3 is a thing, .iteritems() has been removed and .items() actually returns a view object which also doesn't build the full list.

Otherwise, excellent explanation.

2 Likes

If his function works by copying dicts, this should work. I assume deepcopy choked on that first object and the nested ones are handled just fine.
Or they're not actually dicts but the type name contains "dict" and have a copy method... In which case, replace dict in the isinstance check by the actual type, and use .copy() on its items instead of deepcopy, but that seems unlikely.

Alright. I mainly use python3 and went straight to what I know...

1 Like

he is not copying dicts, your function doesnt work :frowning:
image

from com.inductive.ignition.common.script.abc import AbstractMutableJythonMap,AbstractMutableJythonSequence

has to be from

from com.inductiveautomation.ignition.common.script.abc import AbstractMutableJythonMap,AbstractMutableJythonSequence

btw

Then I’m wondering how his original function worked, or rather, I’m wondering what type he was checking for with if str(type(dicOriginen[key])).find("dict") > -1.
I assumed it was a dict, but I guess it might actually be a ‘fake’ one that has “dict” in its name, and which also supports .copy() and .items(), but is not handled by deepcopy