[KED-1513] PickleDataSet error in jupyter notebook

[jfogelberg]

Description

Running a pipeline using

context.run()

inside a notebook, launched using

kedro jupyter notebook

raises exception when saving PickleDataSet.
Running the same pipeline in terminal works fine without any error:

kedro run

Loading the pickle file from inside the notebook works fine:

context.io.load("my_data")

The object is a custom class containing a pandas DataFrame and some metadata.

Context

Currently I cannot run my pipeline from jupyter.

Steps to Reproduce

1. Add PickleDataSet my_data in the catalog.
2. Create a pipeline that saves a custom class object to my_data
3. Run pipeline from jupyter notebook using context.run()

Expected Result

Pipeline runs without error.

Actual Result

Error is raised.

DataSetError: <class 'my_custom_class'> cannot be serialized. PickleDataSet can only be used with serializable data

 _pickle.PicklingError: Can't pickle <class 'my_custom_class'>: it's not the same object as my_custom_class

Your Environment

• Kedro version used (pip show kedro or kedro -V): 0.15.8
• Python version used (python -V): 3.6.8
• Operating system and version: Windows 10
• Error is raised even if I run context.run() as my first cell in the notebook without having imported any other library or magic commands.

[mzjp2]

Thanks for flagging this @jfogelberg, I've added it to our issue tracker and we'll discuss this on Monday. If you have any more information/ideas on how to fix, please feel free to add. :)

[andrii-ivaniuk]

The issue was resolved in af76335 commit.