I have a large object that I want to save to the disk because it takes a minute to generate. The OOM reaper kills the process while pickle.dump ing the object.
It’s a tuple of dicts of tuple of array.array.
Can pickle dump in chunks? If not, is there another technique I can use?
It’s a tuple of dicts of tuples of array.array, no numpy or torch :(
So python standard library lists?