Solution for ValueError in Keras imdb.load_data

Origin

I tried to read with the following code.

from keras.datasets import imdb

(X_train,y_train),(X_test,y_test) = imdb.load_data(num_words=10000)

Then, the following error occurs

ValueError                                Traceback (most recent call last)
<ipython-input-203-2fc6b409cd07> in <module>
      5 #np_load_old = np.load
      6 #np.load = lambda *a,**k: np_load_old(*a, allow_pickle=True, **k)
----> 7 (X_train,y_train),(X_test,y_test) = imdb.load_data(num_words=10000)
      8 #np.load = np_load_old
      9 
/opt/anaconda3/lib/python3.7/site-packages/keras/datasets/imdb.py in load_data(path, num_words, skip_top, maxlen, seed, start_char, oov_char, index_from, **kwargs)
     57                     file_hash='599dadb1135973df5b59232a0e9a887c')
     58     with np.load(path) as f:
---> 59         x_train, labels_train = f['x_train'], f['y_train']
     60         x_test, labels_test = f['x_test'], f['y_test']
     61 
/opt/anaconda3/lib/python3.7/site-packages/numpy/lib/npyio.py in __getitem__(self, key)
    260                 return format.read_array(bytes,
    261                                          allow_pickle=self.allow_pickle,
--> 262                                          pickle_kwargs=self.pickle_kwargs)
    263             else:
    264                 return self.zip.read(key)
/opt/anaconda3/lib/python3.7/site-packages/numpy/lib/format.py in read_array(fp, allow_pickle, pickle_kwargs)
    720         # The array contained Python objects. We need to unpickle the data.
    721         if not allow_pickle:
--> 722             raise ValueError("Object arrays cannot be loaded when "
    723                              "allow_pickle=False")
    724         if pickle_kwargs is None:
ValueError: Object arrays cannot be loaded when allow_pickle=False

Cause

It seems that the cause is that ʻallow_pickle = False` has become the default when upgrading the numpy version.

Solution

It can be handled by lowering the version of numpy, but it is troublesome, so try to solve it in the code.

Specifically, it is solved by replacing it with the following code.

#Specify up to 10000 words in frequency order
from keras.datasets import imdb
​
np_load_old = np.load
np.load = lambda *a,**k: np_load_old(*a, allow_pickle=True, **k)
(X_train,y_train),(X_test,y_test) = imdb.load_data(num_words=10000)
np.load = np_load_old

In np.load, it is temporarily overwritten so that ʻallow_pickle = True is specified. When ʻimdb.load_data is complete, restore the definition of np.load.

reference

• https://teratail.com/questions/186503
• https://stackoverflow.com/questions/55890813/how-to-fix-object-arrays-cannot-be-loaded-when-allow-pickle-false-for-imdb-loa