Error occurred when msgpack loads lmdb

[youcaiSUN]

Hi tengda,

When I test lmdb_dataset.py using your provided lmdb, I got the following error:
Loading LMDB from */UCF101/ucf101_rgb_lmdb/ucf101_frame.lmdb, split:1 Frame Dataset from "*/Code/CoCLR/dataset/../process_data/data/ucf101" has #class 101 filter out too short videos ... Traceback (most recent call last): File "*/Code/CoCLR/dataset/lmdb_dataset.py", line 999, in <module> x = dataset[0] File "*/Code/CoCLR/dataset/lmdb_dataset.py", line 146, in __getitem__ raw = msgpack.loads(txn.get(self.get_video_id[vname].encode('ascii'))) File "msgpack/_unpacker.pyx", line 195, in msgpack._cmsgpack.unpackb UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 0: invalid start byte

The test script is:
if __name__ == "__main__": dataset = UCF101LMDB_2CLIP(db_path=os.path.join(lmdb_root, 'UCF101/ucf101_rgb_lmdb/ucf101_frame.lmdb')) x = dataset[0] print(x)

However, when I use my custom lmdb, there is no error occurred.
The msgpack verison I used is 1.0.1.
Could you help me to figure it out? Thanks!

[TengdaHan]

The package "msgpack" recently updated. I created the lmdb file using msgpack==0.6.2.

To load older version of lmdb with the new msgpack (e.g. 1.0.0+), using raw=True like msgpack.loads(txn.get(self.get_video_id[vname].encode('ascii')), raw=True) should work.
Source link: https://pypi.org/project/msgpack/#description

Let me know if it doesn't work.

[youcaiSUN]

It works! Thanks for your promptly reply!