Merge/update nested dictionary

Hi there,
I bumped into phyton because I’m looking for some ‘simple code’ to do daily routine calculation and I got stuck.
I try to explain what happen with a schematic of the code. Goal to achieve:

  • for each day in a while loop do a calculation with a for cycle which return a dictionary d_for_calc
  • before update while loop counter update a dictionary d_while_calc{date:d_for_calc}

something like:
d_for_calc{}
d_while_calc{}
data=begin_date
while date<end_date
for i …
d_for_calc= calculation
d_while_calc[date]=d_for_calc
date +=1

Result and problem:
at the end of the while loop all the values of d_while_calc are identical to the last d_for_calc and again if I can show schematicly the result at the end of code considering loop of 3 days:
d_while_calc{date1:d_for_calc(date3), date2:d_for_calc(date3), date3:d_for_calc(date3).

I tried several ways including dictionary update, create temp daily dictionary and merge with the main one but with no change.

I think I miss some knowledge of python because I used similar concept wit other languages and it was working

Can someone help me with a correct merge/update nested dictionary procedure?

I hope my description has been good enough or please let me know and I’ll update it with more information.

I’m using anaconda/spyder with python 3.8.5

thanks you all

Without seeing the code it will be hard to help. Rough guess based on the description is that d_for_cal might be using mutable data type and calculations are just mutating it, ending up with all results pointing to a single literally identical object.

I have some trouble understanding the issue.
So you have a couple of days, want to calculate something on each and then add them to a dictionairy. Ok.
Now what is the problem? Sounds straightforward enough.

Please share your code, maybe with an example calculation and your desired result compared to the actual result.

setting few print() statement along the code what I see is exactly what you are describing.
I never come across this mutable data type nor how I’m triggering/using such type.

Can you suggest anything I should be aware causing this issue?

I try to clean the code and post

this is part of the code uses skyfield module and functions

data_calcolo = data_inizio

while data_calcolo <= data_fine:
    t = ts.utc(data_calcolo)
    h= planets['sun'].at(t)
    g= d_pianeti['Earth'].at(t)

####daily calculations

    for j in range(len(l_pianeti)):
        planet_name = l_pianeti[j]
        planet_obs = d_pianeti[planet_name]
        lat, h_long, distance = h.observe(planet_obs).frame_latlon(ef)
        
        # d_data_sid is the dictionary I build for each day (d_for_cal)
        d_data_sid['h_sid'][planet_name]=h_long.degrees
        d_data_sid['h_lat'][planet_name]=lat.degrees

    print('Section 1')
    #d_sid is the d_while_calc
    print(d_sid)
    d_sid.update({data_calcolo:d_data_sid})
    print('Section 2')
    print(d_sin)

the print on Section 1 is the dictionary before the update with current {date:calculation}. In reality the values of it looks all like the new d_data_sid values showing this mutable data type condition

The principle seams straight forward but the result isn’t.
Thanks for helping me

Generally what is happening can be shown on a bit different and much simpler example:

list1 = [1, 2, 3]
list2 = list1
list2[1] = 4
print(list1)
print(list2)

As it can be seen, despite of changing list2 changes are reflected also on list1. That’s because both list1 and list2 are referencing the same object, just using different names.

In the code:

        d_data_sid['h_sid'][planet_name]=h_long.degrees
        d_data_sid['h_lat'][planet_name]=lat.degrees

This doesn’t create new d_data_sid dictionary for the current update, but uses the same one every time, just changing values.

If d_data_sid doesn’t contain anything else than the updated data, then one way dealing with this would be making empty dictionary at the start of update. Otherwise some deeper copy might be needed. But it depends on what exactly is in it, and how it’s created for each day.

Thanks @sanity ! I’m possibly getting toward the solution.

When you say empty the dictionary do you mean using .clear(), in other words:
d_data_sid.clear()

This dictionary only store the daily data. the only issue is that I preformat the dictionary and keys like below:

d_data_sid={'h_sid':{},
            'h_sid_time':{},
            'h_lat':{},
            'g_sid':{},
            'g_sid_time':{},
            'g_decl':{},
            'g_lat':{}}

and when I clear it inside the for loop I delete the formatting.

As newby as basic solution I can think the following options:

  • define again the keys in the FOR loop after clear()

  • forget the pre-formatting and pass the nested dictionary with the keys I’ve choose

  • with a loop empty each individual sub-dictionary

Do you have any other and more ‘proper/professional’ way of doing it? Which of the above would you suggest?

In the meanwhile I try one of the above to check if I’ve got the issue of my code :wink:

cheers

quick update:
i used .clear() and redifine keys in the for loop.

The outcome is that the d_sid (general one which include all days calc) has only the last date:value with numbers like:

d_sid={data_inizio:{},
...
date_x:{},
...
data_fine:{numbers as appropriate

does the option

new_dict = d_data_sid.copy()

would be of help? In other terms, I prepare a copy and then update the general d_sid with the copy?

At the moment after .clear() I’m experiencing some problem which I’m trying to understand but please let me know your suggestion for me to proceed

cheers

HI @sanity ,
at the moment I found one way which is working.

-I create a d_temp=d_data_sid.copy()
-I update the d_sid with d_temp (d_sid is the general dictionary)
-before the end of the while loop: d_data_sid.clear()
-define again the structure with keys of d_data_sid

As I said all the above work but I’m not sure it is the greatest way. If you have smarter way of doing it please advice.

For the moment problem solved and I can focus on next steps

Thanks you a lot

I was thinking about something like:

while data_calcolo <= data_fine:
(...)
d_data_sid = {}
####daily calculations
(...)
d_sid.update({data_calcolo:d_data_sid})

As I understand d_data_sid initially contain just few keys with empty dictionaries as values. So this could be done before daily calculations loop, similarly, as creating d_data_sid as empty dictionary in the example above. Because before each loop new dictionary with dictionaries would be name d_data_sid operations in the loop would be performed on this separate object, which would be later put into the d_sid.

@sanity , I’ll give it a go later on when I’ll revise the sw for a easier reading.

Your way if it works for sure require less resources than mine. At the moment I want to go ahead so I can start to use my calculation as soon as possible;)

Thanks again for your great and help and support!!!

Ciao ciao

This topic was automatically closed 182 days after the last reply. New replies are no longer allowed.