PickleProperty put in Python 2.7 GAE NDB can't be read by Cloud NDB on Python 3 if it contains certain types #909
Description
Discovered while migrating to python 3
python-ndb/google/cloud/ndb/model.py
Lines 3064 to 3065 in b0f4310
Related to #595
The current PickleProperty referenced in the github code link above uses encoding="bytes" when unpickling. This does not work for all data types that was previously pickled with NDB in python 2.7. According to: This post, if you had pickled objects that contained datetime and some others, the only way to unpickle correctly is to use encoding="latin1"
We have internally fixed this with:
`class PicklePropertyFixed(model.BlobProperty):
def _to_base_type(self, value):
return pickle.dumps(value, pickle.HIGHEST_PROTOCOL)
def _from_base_type(self, value):
"""Because the google.cloud.ndb.model.PickleProperty ONLY decodes using the
'bytes' encoding, we must use this custom class to override this.
The reason is because the encoding 'latin1' has to be used if the pickled object
has instances of datetime, date, time, or numpy arrayswhich. Our model uses datetime
"""
if type(value) is bytes:
return pickle.loads(value, encoding="latin1")
return pickle.loads(value)`